== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 04:33:08 (1788769988)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.46 seconds: 21.70 ops/second
 - mkdir 149 (time 1788770008.78 total 10.08 last 14.78)
 - mkdir 318 (time 1788770018.83 total 20.13 last 16.81)
 - mkdir 471 (time 1788770028.85 total 30.16 last 15.26)
 - mkdir 629 (time 1788770038.89 total 40.19 last 15.75)
 - mkdir 789 (time 1788770048.91 total 50.22 last 15.96)
 - mkdir 936 (time 1788770058.96 total 60.26 last 14.63)
total: 1000 mkdir in 64.60 seconds: 15.48 ops/second
 - mkdir 174 (time 1788770011.98 total 10.03 last 17.35)
 - mkdir 336 (time 1788770022.00 total 20.06 last 16.16)
 - mkdir 489 (time 1788770032.01 total 30.06 last 15.29)
 - mkdir 664 (time 1788770042.05 total 40.10 last 17.43)
 - mkdir 821 (time 1788770052.14 total 50.19 last 15.56)
 - mkdir 969 (time 1788770062.18 total 60.24 last 14.73)
total: 1000 mkdir in 63.30 seconds: 15.80 ops/second
 - mkdir 160 (time 1788770011.41 total 10.04 last 15.94)
 - mkdir 321 (time 1788770021.45 total 20.08 last 16.04)
 - mkdir 475 (time 1788770031.48 total 30.10 last 15.36)
 - mkdir 642 (time 1788770041.55 total 40.17 last 16.59)
 - mkdir 799 (time 1788770051.57 total 50.20 last 15.66)
 - mkdir 950 (time 1788770061.62 total 60.25 last 15.03)
total: 1000 mkdir in 65.01 seconds: 15.38 ops/second
 - mkdir 159 (time 1788770010.92 total 10.06 last 15.81)
 - mkdir 323 (time 1788770021.02 total 20.15 last 16.24)
 - mkdir 478 (time 1788770031.04 total 30.18 last 15.47)
 - mkdir 637 (time 1788770041.09 total 40.23 last 15.82)
 - mkdir 793 (time 1788770051.11 total 50.25 last 15.57)
 - mkdir 943 (time 1788770061.23 total 60.36 last 14.83)
total: 1000 mkdir in 65.92 seconds: 15.17 ops/second
 - mkdir 166 (time 1788770011.72 total 10.02 last 16.57)
 - mkdir 327 (time 1788770021.77 total 20.07 last 16.01)
 - mkdir 475 (time 1788770031.81 total 30.11 last 14.75)
 - mkdir 644 (time 1788770041.85 total 40.15 last 16.83)
 - mkdir 803 (time 1788770051.94 total 50.24 last 15.75)
 - mkdir 950 (time 1788770061.96 total 60.26 last 14.67)
total: 1000 mkdir in 65.14 seconds: 15.35 ops/second
 - mkdir 168 (time 1788770012.02 total 10.05 last 16.72)
 - mkdir 328 (time 1788770022.02 total 20.05 last 16.00)
 - mkdir 472 (time 1788770032.05 total 30.08 last 14.36)
 - mkdir 640 (time 1788770042.11 total 40.13 last 16.71)
 - mkdir 790 (time 1788770052.18 total 50.21 last 14.89)
 - mkdir 944 (time 1788770062.22 total 60.25 last 15.34)
total: 1000 mkdir in 66.39 seconds: 15.06 ops/second
 - mkdir 167 (time 1788770012.13 total 10.04 last 16.64)
 - mkdir 322 (time 1788770022.17 total 20.08 last 15.44)
 - mkdir 469 (time 1788770032.19 total 30.09 last 14.68)
 - mkdir 635 (time 1788770042.21 total 40.11 last 16.57)
 - mkdir 786 (time 1788770052.25 total 50.16 last 15.03)
 - mkdir 942 (time 1788770062.28 total 60.18 last 15.56)
total: 1000 mkdir in 66.67 seconds: 15.00 ops/second
 - mkdir 168 (time 1788770012.27 total 10.04 last 16.73)
 - mkdir 329 (time 1788770022.38 total 20.15 last 15.93)
 - mkdir 477 (time 1788770032.45 total 30.22 last 14.69)
 - mkdir 647 (time 1788770042.54 total 40.30 last 16.86)
 - mkdir 797 (time 1788770052.55 total 50.32 last 14.98)
 - mkdir 948 (time 1788770062.58 total 60.35 last 15.05)
total: 1000 mkdir in 66.54 seconds: 15.03 ops/second
 - mkdir 171 (time 1788770012.37 total 10.05 last 17.02)
 - mkdir 332 (time 1788770022.39 total 20.07 last 16.06)
 - mkdir 477 (time 1788770032.43 total 30.11 last 14.45)
 - mkdir 647 (time 1788770042.53 total 40.21 last 16.84)
 - mkdir 793 (time 1788770052.54 total 50.22 last 14.58)
 - mkdir 943 (time 1788770062.55 total 60.23 last 14.97)
total: 1000 mkdir in 66.88 seconds: 14.95 ops/second
 - mkdir 149 (time 1788770011.00 total 10.02 last 14.86)
 - mkdir 303 (time 1788770021.05 total 20.07 last 15.33)
 - mkdir 449 (time 1788770031.10 total 30.12 last 14.52)
 - mkdir 610 (time 1788770041.11 total 40.13 last 16.08)
 - mkdir 766 (time 1788770051.20 total 50.22 last 15.46)
 - mkdir 914 (time 1788770061.23 total 60.26 last 14.75)
total: 1000 mkdir in 68.83 seconds: 14.53 ops/second
Time for file creation 79 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 17 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.51 seconds: 19.53 ops/second
 - mkdir 143 (time 1788770638.38 total 10.01 last 14.29)
 - mkdir 276 (time 1788770648.41 total 20.04 last 13.26)
 - mkdir 424 (time 1788770658.42 total 30.04 last 14.79)
 - mkdir 587 (time 1788770668.50 total 40.13 last 16.16)
 - mkdir 723 (time 1788770678.59 total 50.22 last 13.49)
 - mkdir 875 (time 1788770688.62 total 60.25 last 15.15)
total: 1000 mkdir in 68.95 seconds: 14.50 ops/second
 - mkdir 149 (time 1788770639.06 total 10.06 last 14.81)
 - mkdir 283 (time 1788770649.07 total 20.08 last 13.38)
 - mkdir 432 (time 1788770659.19 total 30.20 last 14.72)
 - mkdir 599 (time 1788770669.23 total 40.24 last 16.64)
 - mkdir 733 (time 1788770679.28 total 50.29 last 13.33)
 - mkdir 886 (time 1788770689.32 total 60.32 last 15.25)
total: 1000 mkdir in 68.36 seconds: 14.63 ops/second
 - mkdir 141 (time 1788770638.20 total 10.01 last 14.08)
 - mkdir 270 (time 1788770648.20 total 20.02 last 12.89)
 - mkdir 412 (time 1788770658.27 total 30.09 last 14.11)
 - mkdir 576 (time 1788770668.33 total 40.14 last 16.31)
 - mkdir 712 (time 1788770678.33 total 50.15 last 13.59)
 - mkdir 864 (time 1788770688.38 total 60.20 last 15.12)
 - mkdir 999 (time 1788770698.40 total 70.22 last 13.48)
total: 1000 mkdir in 70.22 seconds: 14.24 ops/second
 - mkdir 131 (time 1788770637.45 total 10.02 last 13.07)
 - mkdir 262 (time 1788770647.45 total 20.02 last 13.10)
 - mkdir 401 (time 1788770657.46 total 30.03 last 13.89)
 - mkdir 564 (time 1788770667.48 total 40.05 last 16.27)
 - mkdir 706 (time 1788770677.48 total 50.05 last 14.20)
 - mkdir 852 (time 1788770687.52 total 60.09 last 14.54)
 - mkdir 992 (time 1788770697.56 total 70.13 last 13.95)
total: 1000 mkdir in 71.15 seconds: 14.05 ops/second
 - mkdir 141 (time 1788770638.26 total 10.00 last 14.10)
 - mkdir 269 (time 1788770648.33 total 20.08 last 12.70)
 - mkdir 411 (time 1788770658.35 total 30.10 last 14.18)
 - mkdir 573 (time 1788770668.37 total 40.12 last 16.16)
 - mkdir 708 (time 1788770678.39 total 50.14 last 13.47)
 - mkdir 862 (time 1788770688.41 total 60.15 last 15.38)
 - mkdir 994 (time 1788770698.48 total 70.23 last 13.11)
total: 1000 mkdir in 70.98 seconds: 14.09 ops/second
 - mkdir 144 (time 1788770638.71 total 10.07 last 14.30)
 - mkdir 271 (time 1788770648.79 total 20.15 last 12.60)
 - mkdir 413 (time 1788770658.88 total 30.24 last 14.08)
 - mkdir 570 (time 1788770668.89 total 40.24 last 15.69)
 - mkdir 702 (time 1788770678.91 total 50.27 last 13.16)
 - mkdir 857 (time 1788770688.95 total 60.31 last 15.44)
 - mkdir 989 (time 1788770699.07 total 70.42 last 13.05)
total: 1000 mkdir in 71.50 seconds: 13.99 ops/second
 - mkdir 148 (time 1788770639.40 total 10.08 last 14.69)
 - mkdir 279 (time 1788770649.45 total 20.12 last 13.04)
 - mkdir 410 (time 1788770659.48 total 30.15 last 13.06)
 - mkdir 571 (time 1788770669.53 total 40.20 last 16.02)
 - mkdir 700 (time 1788770679.54 total 50.21 last 12.88)
 - mkdir 859 (time 1788770689.65 total 60.32 last 15.73)
 - mkdir 990 (time 1788770699.69 total 70.36 last 13.05)
total: 1000 mkdir in 71.36 seconds: 14.01 ops/second
 - mkdir 121 (time 1788770636.40 total 10.03 last 12.07)
 - mkdir 249 (time 1788770646.41 total 20.04 last 12.79)
 - mkdir 379 (time 1788770656.46 total 30.08 last 12.94)
 - mkdir 530 (time 1788770666.50 total 40.12 last 15.04)
 - mkdir 674 (time 1788770676.59 total 50.21 last 14.27)
 - mkdir 821 (time 1788770686.61 total 60.23 last 14.67)
 - mkdir 959 (time 1788770696.61 total 70.23 last 13.80)
total: 1000 mkdir in 74.75 seconds: 13.38 ops/second
 - mkdir 121 (time 1788770637.20 total 10.06 last 12.03)
 - mkdir 250 (time 1788770647.26 total 20.12 last 12.82)
 - mkdir 385 (time 1788770657.29 total 30.15 last 13.45)
 - mkdir 541 (time 1788770667.37 total 40.23 last 15.48)
 - mkdir 680 (time 1788770677.40 total 50.26 last 13.86)
 - mkdir 830 (time 1788770687.49 total 60.35 last 14.87)
 - mkdir 966 (time 1788770697.49 total 70.35 last 13.60)
total: 1000 mkdir in 74.39 seconds: 13.44 ops/second
 - mkdir 137 (time 1788770638.69 total 10.09 last 13.58)
 - mkdir 259 (time 1788770648.73 total 20.13 last 12.15)
 - mkdir 398 (time 1788770658.75 total 30.16 last 13.86)
 - mkdir 551 (time 1788770668.79 total 40.19 last 15.25)
 - mkdir 685 (time 1788770678.80 total 50.20 last 13.39)
 - mkdir 841 (time 1788770688.85 total 60.26 last 15.51)
 - mkdir 972 (time 1788770699.06 total 70.46 last 12.84)
total: 1000 mkdir in 73.85 seconds: 13.54 ops/second
Time for file creation 87 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788770710 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788770711 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788770711 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788770712 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788770712 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788770712 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788770712 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788770712 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788770712 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1788770713 ; total 1 ; last 1)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
Time for file removal 47 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8