== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 12:49:16 (1787849356)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.49 seconds: 20.46 ops/second
 - mkdir 169 (time 1787849376.71 total 10.04 last 16.84)
 - mkdir 329 (time 1787849386.72 total 20.04 last 15.99)
 - mkdir 487 (time 1787849396.72 total 30.05 last 15.79)
 - mkdir 646 (time 1787849406.79 total 40.12 last 15.79)
 - mkdir 815 (time 1787849416.81 total 50.14 last 16.87)
 - mkdir 976 (time 1787849426.82 total 60.15 last 16.08)
total: 1000 mkdir in 61.53 seconds: 16.25 ops/second
 - mkdir 172 (time 1787849376.10 total 10.01 last 17.18)
 - mkdir 328 (time 1787849386.13 total 20.04 last 15.55)
 - mkdir 483 (time 1787849396.17 total 30.08 last 15.44)
 - mkdir 641 (time 1787849406.27 total 40.18 last 15.65)
 - mkdir 801 (time 1787849416.29 total 50.20 last 15.97)
 - mkdir 961 (time 1787849426.30 total 60.21 last 15.97)
total: 1000 mkdir in 63.39 seconds: 15.77 ops/second
 - mkdir 169 (time 1787849376.84 total 10.04 last 16.83)
 - mkdir 326 (time 1787849386.91 total 20.11 last 15.59)
 - mkdir 483 (time 1787849396.93 total 30.13 last 15.67)
 - mkdir 645 (time 1787849406.96 total 40.17 last 16.14)
 - mkdir 810 (time 1787849416.99 total 50.19 last 16.46)
 - mkdir 966 (time 1787849427.01 total 60.21 last 15.56)
total: 1000 mkdir in 62.88 seconds: 15.90 ops/second
 - mkdir 166 (time 1787849376.59 total 10.04 last 16.53)
 - mkdir 320 (time 1787849386.62 total 20.07 last 15.35)
 - mkdir 474 (time 1787849396.70 total 30.15 last 15.28)
 - mkdir 639 (time 1787849406.78 total 40.22 last 16.37)
 - mkdir 799 (time 1787849416.80 total 50.25 last 15.96)
 - mkdir 962 (time 1787849426.84 total 60.29 last 16.24)
total: 1000 mkdir in 63.65 seconds: 15.71 ops/second
 - mkdir 164 (time 1787849375.96 total 10.02 last 16.36)
 - mkdir 315 (time 1787849386.00 total 20.07 last 15.03)
 - mkdir 471 (time 1787849396.03 total 30.10 last 15.55)
 - mkdir 627 (time 1787849406.05 total 40.12 last 15.57)
 - mkdir 788 (time 1787849416.08 total 50.15 last 16.05)
 - mkdir 950 (time 1787849426.13 total 60.19 last 16.13)
total: 1000 mkdir in 64.30 seconds: 15.55 ops/second
 - mkdir 167 (time 1787849376.89 total 10.02 last 16.67)
 - mkdir 324 (time 1787849386.94 total 20.07 last 15.62)
 - mkdir 478 (time 1787849397.02 total 30.15 last 15.28)
 - mkdir 634 (time 1787849407.06 total 40.19 last 15.54)
 - mkdir 796 (time 1787849417.09 total 50.22 last 16.14)
 - mkdir 955 (time 1787849427.12 total 60.25 last 15.86)
total: 1000 mkdir in 64.08 seconds: 15.61 ops/second
 - mkdir 165 (time 1787849376.52 total 10.03 last 16.46)
 - mkdir 318 (time 1787849386.53 total 20.03 last 15.29)
 - mkdir 472 (time 1787849396.65 total 30.16 last 15.21)
 - mkdir 628 (time 1787849406.77 total 40.28 last 15.42)
 - mkdir 789 (time 1787849416.79 total 50.29 last 16.08)
 - mkdir 951 (time 1787849426.89 total 60.39 last 16.04)
total: 1000 mkdir in 64.53 seconds: 15.50 ops/second
 - mkdir 167 (time 1787849377.02 total 10.02 last 16.66)
 - mkdir 318 (time 1787849387.05 total 20.05 last 15.06)
 - mkdir 475 (time 1787849397.06 total 30.07 last 15.67)
 - mkdir 630 (time 1787849407.09 total 40.09 last 15.47)
 - mkdir 794 (time 1787849417.11 total 50.11 last 16.36)
 - mkdir 948 (time 1787849427.13 total 60.13 last 15.37)
total: 1000 mkdir in 65.20 seconds: 15.34 ops/second
 - mkdir 165 (time 1787849375.10 total 10.00 last 16.50)
 - mkdir 320 (time 1787849385.11 total 20.02 last 15.47)
 - mkdir 466 (time 1787849395.14 total 30.05 last 14.55)
 - mkdir 615 (time 1787849405.19 total 40.10 last 14.83)
 - mkdir 771 (time 1787849415.21 total 50.11 last 15.58)
 - mkdir 924 (time 1787849425.28 total 60.18 last 15.20)
total: 1000 mkdir in 67.18 seconds: 14.89 ops/second
 - mkdir 165 (time 1787849376.71 total 10.07 last 16.38)
 - mkdir 319 (time 1787849386.79 total 20.15 last 15.27)
 - mkdir 469 (time 1787849396.82 total 30.18 last 14.96)
 - mkdir 620 (time 1787849406.83 total 40.19 last 15.08)
 - mkdir 781 (time 1787849416.90 total 50.26 last 15.99)
 - mkdir 940 (time 1787849427.05 total 60.41 last 15.66)
total: 1000 mkdir in 66.23 seconds: 15.10 ops/second
Time for file creation 76 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 12 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.54 seconds: 18.57 ops/second
 - mkdir 145 (time 1787849982.16 total 10.03 last 14.45)
 - mkdir 278 (time 1787849992.20 total 20.07 last 13.25)
 - mkdir 408 (time 1787850002.20 total 30.07 last 13.00)
 - mkdir 547 (time 1787850012.24 total 40.11 last 13.85)
 - mkdir 697 (time 1787850022.29 total 50.16 last 14.93)
 - mkdir 843 (time 1787850032.32 total 60.19 last 14.55)
 - mkdir 979 (time 1787850042.35 total 70.22 last 13.56)
total: 1000 mkdir in 71.59 seconds: 13.97 ops/second
 - mkdir 136 (time 1787849982.60 total 10.09 last 13.48)
 - mkdir 265 (time 1787849992.62 total 20.11 last 12.88)
 - mkdir 399 (time 1787850002.70 total 30.19 last 13.29)
 - mkdir 537 (time 1787850012.75 total 40.24 last 13.73)
 - mkdir 690 (time 1787850022.80 total 50.29 last 15.22)
 - mkdir 832 (time 1787850032.91 total 60.40 last 14.04)
 - mkdir 975 (time 1787850043.00 total 70.49 last 14.17)
total: 1000 mkdir in 73.02 seconds: 13.69 ops/second
 - mkdir 141 (time 1787849982.83 total 10.04 last 14.04)
 - mkdir 271 (time 1787849992.90 total 20.11 last 12.91)
 - mkdir 402 (time 1787850002.93 total 30.14 last 13.07)
 - mkdir 540 (time 1787850012.96 total 40.17 last 13.75)
 - mkdir 686 (time 1787850023.00 total 50.21 last 14.55)
 - mkdir 835 (time 1787850033.06 total 60.27 last 14.82)
 - mkdir 972 (time 1787850043.12 total 70.33 last 13.61)
total: 1000 mkdir in 72.91 seconds: 13.72 ops/second
 - mkdir 145 (time 1787849983.29 total 10.09 last 14.37)
 - mkdir 272 (time 1787849993.38 total 20.18 last 12.59)
 - mkdir 410 (time 1787850003.38 total 30.18 last 13.80)
 - mkdir 549 (time 1787850013.47 total 40.26 last 13.78)
 - mkdir 696 (time 1787850023.59 total 50.39 last 14.52)
 - mkdir 838 (time 1787850033.65 total 60.45 last 14.11)
 - mkdir 981 (time 1787850043.68 total 70.48 last 14.27)
total: 1000 mkdir in 72.60 seconds: 13.77 ops/second
 - mkdir 135 (time 1787849982.54 total 10.02 last 13.47)
 - mkdir 266 (time 1787849992.56 total 20.04 last 13.07)
 - mkdir 399 (time 1787850002.59 total 30.08 last 13.25)
 - mkdir 527 (time 1787850012.60 total 40.09 last 12.80)
 - mkdir 677 (time 1787850022.66 total 50.15 last 14.91)
 - mkdir 817 (time 1787850032.68 total 60.17 last 13.97)
 - mkdir 960 (time 1787850042.71 total 70.20 last 14.26)
total: 1000 mkdir in 74.07 seconds: 13.50 ops/second
 - mkdir 145 (time 1787849983.38 total 10.01 last 14.48)
 - mkdir 274 (time 1787849993.38 total 20.02 last 12.90)
 - mkdir 404 (time 1787850003.46 total 30.10 last 12.90)
 - mkdir 543 (time 1787850013.47 total 40.11 last 13.88)
 - mkdir 691 (time 1787850023.51 total 50.15 last 14.74)
 - mkdir 830 (time 1787850033.55 total 60.19 last 13.85)
 - mkdir 970 (time 1787850043.59 total 70.23 last 13.95)
total: 1000 mkdir in 73.94 seconds: 13.53 ops/second
 - mkdir 145 (time 1787849983.75 total 10.07 last 14.40)
 - mkdir 275 (time 1787849993.75 total 20.07 last 13.00)
 - mkdir 405 (time 1787850003.75 total 30.07 last 13.00)
 - mkdir 544 (time 1787850013.75 total 40.08 last 13.89)
 - mkdir 689 (time 1787850023.77 total 50.09 last 14.48)
 - mkdir 827 (time 1787850033.81 total 60.13 last 13.75)
 - mkdir 972 (time 1787850043.86 total 70.19 last 14.42)
total: 1000 mkdir in 73.81 seconds: 13.55 ops/second
 - mkdir 136 (time 1787849983.52 total 10.04 last 13.55)
 - mkdir 265 (time 1787849993.58 total 20.10 last 12.81)
 - mkdir 404 (time 1787850003.62 total 30.14 last 13.85)
 - mkdir 543 (time 1787850013.64 total 40.16 last 13.88)
 - mkdir 687 (time 1787850023.64 total 50.16 last 14.40)
 - mkdir 828 (time 1787850033.74 total 60.26 last 13.96)
 - mkdir 969 (time 1787850043.75 total 70.27 last 14.08)
total: 1000 mkdir in 74.12 seconds: 13.49 ops/second
 - mkdir 133 (time 1787849983.34 total 10.05 last 13.24)
 - mkdir 253 (time 1787849993.39 total 20.10 last 11.94)
 - mkdir 384 (time 1787850003.50 total 30.21 last 12.96)
 - mkdir 520 (time 1787850013.53 total 40.23 last 13.57)
 - mkdir 662 (time 1787850023.59 total 50.29 last 14.12)
 - mkdir 803 (time 1787850033.63 total 60.34 last 14.04)
 - mkdir 944 (time 1787850043.64 total 70.34 last 14.10)
total: 1000 mkdir in 76.94 seconds: 13.00 ops/second
 - mkdir 129 (time 1787849981.40 total 10.01 last 12.89)
 - mkdir 255 (time 1787849991.53 total 20.13 last 12.45)
 - mkdir 375 (time 1787850001.55 total 30.15 last 11.97)
 - mkdir 498 (time 1787850011.56 total 40.16 last 12.29)
 - mkdir 635 (time 1787850021.69 total 50.29 last 13.53)
 - mkdir 768 (time 1787850031.77 total 60.37 last 13.20)
 - mkdir 903 (time 1787850041.80 total 70.40 last 13.46)
total: 1000 mkdir in 79.51 seconds: 12.58 ops/second
Time for file creation 88 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787850059 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1787850059 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787850060 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787850060 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787850061 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787850060 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787850060 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787850060 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787850061 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787850060 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
Time for file removal 49 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8