== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 03:38:13 (1788766693)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.45 seconds: 22.22 ops/second
 - mkdir 139 (time 1788766719.31 total 10.09 last 13.78)
 - mkdir 283 (time 1788766729.33 total 20.11 last 14.38)
 - mkdir 448 (time 1788766739.39 total 30.17 last 16.39)
 - mkdir 589 (time 1788766749.42 total 40.19 last 14.07)
 - mkdir 739 (time 1788766759.47 total 50.25 last 14.92)
 - mkdir 877 (time 1788766769.51 total 60.29 last 13.75)
total: 1000 mkdir in 65.58 seconds: 15.25 ops/second
 - mkdir 128 (time 1788766718.38 total 10.05 last 12.74)
 - mkdir 270 (time 1788766728.38 total 20.05 last 14.19)
 - mkdir 435 (time 1788766738.39 total 30.06 last 16.49)
 - mkdir 564 (time 1788766748.40 total 40.07 last 12.88)
 - mkdir 717 (time 1788766758.52 total 50.19 last 15.12)
 - mkdir 842 (time 1788766768.54 total 60.21 last 12.47)
total: 1000 mkdir in 68.37 seconds: 14.63 ops/second
 - mkdir 134 (time 1788766719.38 total 10.04 last 13.35)
 - mkdir 269 (time 1788766729.38 total 20.04 last 13.49)
 - mkdir 434 (time 1788766739.41 total 30.07 last 16.45)
 - mkdir 567 (time 1788766749.51 total 40.16 last 13.18)
 - mkdir 716 (time 1788766759.51 total 50.17 last 14.89)
 - mkdir 848 (time 1788766769.53 total 60.19 last 13.18)
total: 1000 mkdir in 69.10 seconds: 14.47 ops/second
 - mkdir 120 (time 1788766718.34 total 10.09 last 11.89)
 - mkdir 258 (time 1788766728.35 total 20.10 last 13.78)
 - mkdir 417 (time 1788766738.38 total 30.13 last 15.86)
 - mkdir 547 (time 1788766748.43 total 40.19 last 12.93)
 - mkdir 702 (time 1788766758.55 total 50.30 last 15.32)
 - mkdir 835 (time 1788766768.59 total 60.34 last 13.25)
 - mkdir 996 (time 1788766778.71 total 70.47 last 15.91)
total: 1000 mkdir in 71.19 seconds: 14.05 ops/second
 - mkdir 135 (time 1788766719.59 total 10.03 last 13.46)
 - mkdir 275 (time 1788766729.64 total 20.08 last 13.93)
 - mkdir 435 (time 1788766739.65 total 30.09 last 15.98)
 - mkdir 571 (time 1788766749.67 total 40.11 last 13.57)
 - mkdir 714 (time 1788766759.79 total 50.23 last 14.12)
 - mkdir 847 (time 1788766769.82 total 60.26 last 13.27)
 - mkdir 997 (time 1788766779.97 total 70.41 last 14.78)
total: 1000 mkdir in 70.74 seconds: 14.14 ops/second
 - mkdir 128 (time 1788766719.43 total 10.05 last 12.73)
 - mkdir 262 (time 1788766729.47 total 20.10 last 13.34)
 - mkdir 420 (time 1788766739.49 total 30.12 last 15.77)
 - mkdir 552 (time 1788766749.50 total 40.13 last 13.18)
 - mkdir 701 (time 1788766759.55 total 50.18 last 14.82)
 - mkdir 836 (time 1788766769.63 total 60.25 last 13.40)
 - mkdir 990 (time 1788766779.72 total 70.34 last 15.26)
total: 1000 mkdir in 71.54 seconds: 13.98 ops/second
 - mkdir 122 (time 1788766718.88 total 10.06 last 12.12)
 - mkdir 265 (time 1788766728.96 total 20.14 last 14.20)
 - mkdir 420 (time 1788766738.98 total 30.16 last 15.46)
 - mkdir 548 (time 1788766749.01 total 40.19 last 12.76)
 - mkdir 692 (time 1788766759.08 total 50.26 last 14.30)
 - mkdir 819 (time 1788766769.11 total 60.29 last 12.67)
 - mkdir 981 (time 1788766779.17 total 70.35 last 16.10)
total: 1000 mkdir in 72.68 seconds: 13.76 ops/second
 - mkdir 119 (time 1788766718.31 total 10.07 last 11.81)
 - mkdir 253 (time 1788766728.31 total 20.08 last 13.40)
 - mkdir 413 (time 1788766738.38 total 30.14 last 15.89)
 - mkdir 545 (time 1788766748.43 total 40.20 last 13.13)
 - mkdir 690 (time 1788766758.46 total 50.22 last 14.46)
 - mkdir 817 (time 1788766768.50 total 60.27 last 12.64)
 - mkdir 977 (time 1788766778.51 total 70.27 last 15.99)
total: 1000 mkdir in 73.38 seconds: 13.63 ops/second
 - mkdir 116 (time 1788766717.77 total 10.04 last 11.56)
 - mkdir 248 (time 1788766727.80 total 20.07 last 13.16)
 - mkdir 406 (time 1788766737.89 total 30.16 last 15.66)
 - mkdir 536 (time 1788766747.90 total 40.17 last 12.98)
 - mkdir 684 (time 1788766757.98 total 50.24 last 14.69)
 - mkdir 806 (time 1788766768.03 total 60.29 last 12.14)
 - mkdir 974 (time 1788766778.13 total 70.39 last 16.64)
total: 1000 mkdir in 73.96 seconds: 13.52 ops/second
 - mkdir 112 (time 1788766717.05 total 10.04 last 11.16)
 - mkdir 236 (time 1788766727.09 total 20.08 last 12.36)
 - mkdir 391 (time 1788766737.09 total 30.08 last 15.49)
 - mkdir 516 (time 1788766747.14 total 40.13 last 12.44)
 - mkdir 665 (time 1788766757.15 total 50.14 last 14.88)
 - mkdir 780 (time 1788766767.19 total 60.18 last 11.46)
 - mkdir 950 (time 1788766777.28 total 70.27 last 16.85)
total: 1000 mkdir in 76.08 seconds: 13.14 ops/second
Time for file creation 87 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 16 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.49 seconds: 20.47 ops/second
 - mkdir 154 (time 1788767355.21 total 10.03 last 15.36)
 - mkdir 309 (time 1788767365.36 total 20.18 last 15.27)
 - mkdir 444 (time 1788767375.36 total 30.18 last 13.50)
 - mkdir 591 (time 1788767385.44 total 40.26 last 14.58)
 - mkdir 765 (time 1788767395.47 total 50.29 last 17.35)
 - mkdir 931 (time 1788767405.48 total 60.31 last 16.57)
total: 1000 mkdir in 64.27 seconds: 15.56 ops/second
 - mkdir 168 (time 1788767356.81 total 10.00 last 16.80)
 - mkdir 301 (time 1788767366.84 total 20.03 last 13.26)
 - mkdir 453 (time 1788767376.90 total 30.09 last 15.10)
 - mkdir 611 (time 1788767386.94 total 40.13 last 15.75)
 - mkdir 776 (time 1788767396.94 total 50.13 last 16.50)
 - mkdir 949 (time 1788767406.98 total 60.17 last 17.23)
total: 1000 mkdir in 62.86 seconds: 15.91 ops/second
 - mkdir 178 (time 1788767357.89 total 10.06 last 17.69)
 - mkdir 309 (time 1788767367.90 total 20.08 last 13.07)
 - mkdir 458 (time 1788767378.05 total 30.23 last 14.68)
 - mkdir 622 (time 1788767388.07 total 40.25 last 16.37)
 - mkdir 788 (time 1788767398.07 total 50.25 last 16.60)
 - mkdir 957 (time 1788767408.08 total 60.26 last 16.88)
total: 1000 mkdir in 62.87 seconds: 15.91 ops/second
 - mkdir 175 (time 1788767359.24 total 10.02 last 17.46)
 - mkdir 306 (time 1788767369.27 total 20.05 last 13.07)
 - mkdir 454 (time 1788767379.29 total 30.07 last 14.76)
 - mkdir 623 (time 1788767389.32 total 40.10 last 16.86)
 - mkdir 800 (time 1788767399.33 total 50.11 last 17.68)
 - mkdir 981 (time 1788767409.36 total 60.14 last 18.04)
total: 1000 mkdir in 61.64 seconds: 16.22 ops/second
 - mkdir 176 (time 1788767358.00 total 10.00 last 17.60)
 - mkdir 306 (time 1788767368.05 total 20.06 last 12.93)
 - mkdir 451 (time 1788767378.06 total 30.06 last 14.49)
 - mkdir 616 (time 1788767388.10 total 40.11 last 16.43)
 - mkdir 777 (time 1788767398.13 total 50.14 last 16.05)
 - mkdir 948 (time 1788767408.18 total 60.19 last 17.02)
total: 1000 mkdir in 63.13 seconds: 15.84 ops/second
 - mkdir 181 (time 1788767359.03 total 10.06 last 17.98)
 - mkdir 313 (time 1788767369.11 total 20.14 last 13.10)
 - mkdir 453 (time 1788767379.14 total 30.17 last 13.96)
 - mkdir 612 (time 1788767389.14 total 40.17 last 15.90)
 - mkdir 780 (time 1788767399.18 total 50.22 last 16.72)
 - mkdir 956 (time 1788767409.20 total 60.23 last 17.57)
total: 1000 mkdir in 63.76 seconds: 15.68 ops/second
 - mkdir 176 (time 1788767359.38 total 10.04 last 17.52)
 - mkdir 308 (time 1788767369.39 total 20.05 last 13.19)
 - mkdir 456 (time 1788767379.41 total 30.07 last 14.77)
 - mkdir 617 (time 1788767389.42 total 40.08 last 16.08)
 - mkdir 782 (time 1788767399.46 total 50.12 last 16.44)
 - mkdir 950 (time 1788767409.47 total 60.14 last 16.78)
total: 1000 mkdir in 65.24 seconds: 15.33 ops/second
 - mkdir 172 (time 1788767359.10 total 10.01 last 17.19)
 - mkdir 297 (time 1788767369.15 total 20.06 last 12.43)
 - mkdir 437 (time 1788767379.22 total 30.13 last 13.91)
 - mkdir 601 (time 1788767389.25 total 40.16 last 16.35)
 - mkdir 766 (time 1788767399.30 total 50.21 last 16.42)
 - mkdir 938 (time 1788767409.32 total 60.23 last 17.17)
total: 1000 mkdir in 66.30 seconds: 15.08 ops/second
 - mkdir 169 (time 1788767358.50 total 10.05 last 16.82)
 - mkdir 294 (time 1788767368.54 total 20.08 last 12.45)
 - mkdir 435 (time 1788767378.59 total 30.14 last 14.03)
 - mkdir 592 (time 1788767388.60 total 40.14 last 15.69)
 - mkdir 754 (time 1788767398.64 total 50.19 last 16.13)
 - mkdir 919 (time 1788767408.65 total 60.19 last 16.49)
total: 1000 mkdir in 67.09 seconds: 14.91 ops/second
 - mkdir 166 (time 1788767359.42 total 10.07 last 16.49)
 - mkdir 289 (time 1788767369.48 total 20.13 last 12.23)
 - mkdir 423 (time 1788767379.53 total 30.18 last 13.33)
 - mkdir 580 (time 1788767389.58 total 40.23 last 15.62)
 - mkdir 738 (time 1788767399.64 total 50.29 last 15.70)
 - mkdir 909 (time 1788767409.66 total 60.31 last 17.07)
total: 1000 mkdir in 68.24 seconds: 14.65 ops/second
Time for file creation 78 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788767425 ; total 0 ; last 0)
total: 1000 unlinks in 26 seconds: 38.461540 unlinks/second
 - unlinked 0 (time 1788767426 ; total 0 ; last 0)
total: 1000 unlinks in 26 seconds: 38.461540 unlinks/second
 - unlinked 0 (time 1788767426 ; total 0 ; last 0)
total: 1000 unlinks in 26 seconds: 38.461540 unlinks/second
 - unlinked 0 (time 1788767427 ; total 0 ; last 0)
total: 1000 unlinks in 25 seconds: 40.000000 unlinks/second
 - unlinked 0 (time 1788767427 ; total 0 ; last 0)
total: 1000 unlinks in 26 seconds: 38.461540 unlinks/second
 - unlinked 0 (time 1788767427 ; total 0 ; last 0)
total: 1000 unlinks in 26 seconds: 38.461540 unlinks/second
 - unlinked 0 (time 1788767427 ; total 0 ; last 0)
total: 1000 unlinks in 27 seconds: 37.037037 unlinks/second
 - unlinked 0 (time 1788767428 ; total 0 ; last 0)
total: 1000 unlinks in 26 seconds: 38.461540 unlinks/second
 - unlinked 0 (time 1788767428 ; total 0 ; last 0)
total: 1000 unlinks in 26 seconds: 38.461540 unlinks/second
 - unlinked 0 (time 1788767428 ; total 0 ; last 0)
total: 1000 unlinks in 27 seconds: 37.037037 unlinks/second
Time for file removal 39 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8