== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 16:18:42 (1788293922)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.55 seconds: 18.07 ops/second
 - mkdir 145 (time 1788293943.80 total 10.04 last 14.44)
 - mkdir 290 (time 1788293953.81 total 20.05 last 14.49)
 - mkdir 438 (time 1788293963.83 total 30.08 last 14.77)
 - mkdir 593 (time 1788293973.90 total 40.15 last 15.39)
 - mkdir 699 (time 1788293984.01 total 50.25 last 10.49)
 - mkdir 820 (time 1788293994.07 total 60.31 last 12.03)
 - mkdir 957 (time 1788294004.07 total 70.32 last 13.69)
total: 1000 mkdir in 72.21 seconds: 13.85 ops/second
 - mkdir 140 (time 1788293944.78 total 10.01 last 13.99)
 - mkdir 278 (time 1788293954.88 total 20.11 last 13.66)
 - mkdir 439 (time 1788293964.91 total 30.14 last 16.06)
 - mkdir 581 (time 1788293975.02 total 40.25 last 14.04)
 - mkdir 699 (time 1788293985.11 total 50.34 last 11.70)
 - mkdir 821 (time 1788293995.18 total 60.41 last 12.11)
 - mkdir 969 (time 1788294005.23 total 70.46 last 14.73)
total: 1000 mkdir in 72.91 seconds: 13.72 ops/second
 - mkdir 136 (time 1788293944.78 total 10.03 last 13.55)
 - mkdir 269 (time 1788293954.80 total 20.06 last 13.27)
 - mkdir 433 (time 1788293964.81 total 30.06 last 16.39)
 - mkdir 581 (time 1788293974.90 total 40.15 last 14.67)
 - mkdir 690 (time 1788293984.90 total 50.16 last 10.89)
 - mkdir 810 (time 1788293994.92 total 60.17 last 11.98)
 - mkdir 960 (time 1788294004.94 total 70.19 last 14.97)
total: 1000 mkdir in 73.51 seconds: 13.60 ops/second
 - mkdir 137 (time 1788293943.54 total 10.02 last 13.67)
 - mkdir 271 (time 1788293953.55 total 20.03 last 13.39)
 - mkdir 419 (time 1788293963.61 total 30.09 last 14.71)
 - mkdir 569 (time 1788293973.64 total 40.12 last 14.95)
 - mkdir 676 (time 1788293983.70 total 50.18 last 10.64)
 - mkdir 791 (time 1788293993.79 total 60.28 last 11.39)
 - mkdir 928 (time 1788294003.80 total 70.28 last 13.70)
total: 1000 mkdir in 76.39 seconds: 13.09 ops/second
 - mkdir 140 (time 1788293944.32 total 10.09 last 13.88)
 - mkdir 277 (time 1788293954.40 total 20.17 last 13.59)
 - mkdir 431 (time 1788293964.42 total 30.19 last 15.37)
 - mkdir 574 (time 1788293974.50 total 40.27 last 14.18)
 - mkdir 679 (time 1788293984.54 total 50.31 last 10.46)
 - mkdir 799 (time 1788293994.54 total 60.31 last 11.99)
 - mkdir 934 (time 1788294004.55 total 70.32 last 13.49)
total: 1000 mkdir in 76.20 seconds: 13.12 ops/second
 - mkdir 132 (time 1788293944.34 total 10.04 last 13.15)
 - mkdir 267 (time 1788293954.43 total 20.13 last 13.38)
 - mkdir 420 (time 1788293964.49 total 30.18 last 15.22)
 - mkdir 567 (time 1788293974.49 total 40.19 last 14.69)
 - mkdir 674 (time 1788293984.54 total 50.23 last 10.65)
 - mkdir 792 (time 1788293994.54 total 60.24 last 11.79)
 - mkdir 930 (time 1788294004.55 total 70.25 last 13.78)
total: 1000 mkdir in 76.71 seconds: 13.04 ops/second
 - mkdir 138 (time 1788293944.82 total 10.00 last 13.79)
 - mkdir 272 (time 1788293954.85 total 20.04 last 13.36)
 - mkdir 427 (time 1788293964.87 total 30.06 last 15.46)
 - mkdir 567 (time 1788293974.98 total 40.17 last 13.85)
 - mkdir 676 (time 1788293985.11 total 50.30 last 10.77)
 - mkdir 793 (time 1788293995.12 total 60.31 last 11.69)
 - mkdir 945 (time 1788294005.17 total 70.36 last 15.12)
total: 1000 mkdir in 76.68 seconds: 13.04 ops/second
 - mkdir 137 (time 1788293944.85 total 10.05 last 13.64)
 - mkdir 273 (time 1788293954.93 total 20.13 last 13.49)
 - mkdir 424 (time 1788293964.94 total 30.13 last 15.09)
 - mkdir 567 (time 1788293975.00 total 40.19 last 14.21)
 - mkdir 675 (time 1788293985.04 total 50.24 last 10.75)
 - mkdir 793 (time 1788293995.08 total 60.28 last 11.75)
 - mkdir 940 (time 1788294005.13 total 70.33 last 14.63)
total: 1000 mkdir in 77.21 seconds: 12.95 ops/second
 - mkdir 135 (time 1788293944.79 total 10.05 last 13.43)
 - mkdir 257 (time 1788293954.85 total 20.11 last 12.13)
 - mkdir 406 (time 1788293964.86 total 30.11 last 14.89)
 - mkdir 547 (time 1788293974.90 total 40.16 last 14.04)
 - mkdir 655 (time 1788293985.02 total 50.27 last 10.67)
 - mkdir 763 (time 1788293995.06 total 60.32 last 10.75)
 - mkdir 908 (time 1788294005.07 total 70.33 last 14.49)
 - mkdir 992 (time 1788294015.08 total 80.33 last 8.39)
total: 1000 mkdir in 81.02 seconds: 12.34 ops/second
 - mkdir 128 (time 1788293944.73 total 10.08 last 12.70)
 - mkdir 250 (time 1788293954.75 total 20.09 last 12.18)
 - mkdir 398 (time 1788293964.78 total 30.13 last 14.75)
 - mkdir 536 (time 1788293974.85 total 40.19 last 13.72)
 - mkdir 640 (time 1788293984.88 total 50.23 last 10.36)
 - mkdir 750 (time 1788293994.94 total 60.28 last 10.94)
 - mkdir 897 (time 1788294004.97 total 70.31 last 14.66)
 - mkdir 985 (time 1788294015.10 total 80.45 last 8.68)
total: 1000 mkdir in 81.87 seconds: 12.21 ops/second
Time for file creation 92 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 17 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.56 seconds: 17.99 ops/second
 - mkdir 145 (time 1788294627.29 total 10.04 last 14.44)
 - mkdir 302 (time 1788294637.32 total 20.07 last 15.65)
 - mkdir 446 (time 1788294647.35 total 30.10 last 14.36)
 - mkdir 585 (time 1788294657.39 total 40.15 last 13.84)
 - mkdir 726 (time 1788294667.45 total 50.21 last 14.02)
 - mkdir 874 (time 1788294677.51 total 60.26 last 14.72)
total: 1000 mkdir in 68.62 seconds: 14.57 ops/second
 - mkdir 141 (time 1788294627.67 total 10.06 last 14.02)
 - mkdir 291 (time 1788294637.78 total 20.17 last 14.83)
 - mkdir 436 (time 1788294647.78 total 30.18 last 14.49)
 - mkdir 572 (time 1788294657.86 total 40.25 last 13.49)
 - mkdir 714 (time 1788294667.88 total 50.27 last 14.18)
 - mkdir 860 (time 1788294677.90 total 60.29 last 14.57)
total: 1000 mkdir in 69.89 seconds: 14.31 ops/second
 - mkdir 143 (time 1788294628.77 total 10.02 last 14.27)
 - mkdir 297 (time 1788294638.80 total 20.04 last 15.36)
 - mkdir 448 (time 1788294648.91 total 30.16 last 14.92)
 - mkdir 583 (time 1788294658.98 total 40.23 last 13.41)
 - mkdir 726 (time 1788294668.98 total 50.23 last 14.30)
 - mkdir 871 (time 1788294679.01 total 60.26 last 14.45)
total: 1000 mkdir in 69.77 seconds: 14.33 ops/second
 - mkdir 141 (time 1788294628.52 total 10.00 last 14.10)
 - mkdir 289 (time 1788294638.60 total 20.09 last 14.68)
 - mkdir 435 (time 1788294648.61 total 30.10 last 14.58)
 - mkdir 572 (time 1788294658.71 total 40.19 last 13.57)
 - mkdir 721 (time 1788294668.71 total 50.19 last 14.90)
 - mkdir 865 (time 1788294678.72 total 60.21 last 14.38)
total: 1000 mkdir in 70.13 seconds: 14.26 ops/second
 - mkdir 141 (time 1788294628.74 total 10.02 last 14.08)
 - mkdir 288 (time 1788294638.80 total 20.07 last 14.62)
 - mkdir 430 (time 1788294648.81 total 30.09 last 14.18)
 - mkdir 560 (time 1788294658.84 total 40.11 last 12.96)
 - mkdir 704 (time 1788294668.88 total 50.15 last 14.35)
 - mkdir 848 (time 1788294678.90 total 60.17 last 14.37)
 - mkdir 979 (time 1788294688.95 total 70.22 last 13.03)
total: 1000 mkdir in 73.17 seconds: 13.67 ops/second
 - mkdir 150 (time 1788294629.41 total 10.07 last 14.89)
 - mkdir 296 (time 1788294639.48 total 20.14 last 14.50)
 - mkdir 437 (time 1788294649.53 total 30.19 last 14.04)
 - mkdir 567 (time 1788294659.62 total 40.28 last 12.88)
 - mkdir 713 (time 1788294669.67 total 50.33 last 14.53)
 - mkdir 856 (time 1788294679.71 total 60.37 last 14.24)
 - mkdir 984 (time 1788294689.85 total 70.51 last 12.63)
total: 1000 mkdir in 72.76 seconds: 13.74 ops/second
 - mkdir 149 (time 1788294629.42 total 10.06 last 14.81)
 - mkdir 293 (time 1788294639.49 total 20.14 last 14.29)
 - mkdir 435 (time 1788294649.53 total 30.17 last 14.15)
 - mkdir 567 (time 1788294659.65 total 40.30 last 13.04)
 - mkdir 716 (time 1788294669.72 total 50.36 last 14.80)
 - mkdir 861 (time 1788294679.76 total 60.41 last 14.43)
 - mkdir 985 (time 1788294689.91 total 70.55 last 12.22)
total: 1000 mkdir in 72.83 seconds: 13.73 ops/second
 - mkdir 142 (time 1788294629.50 total 10.03 last 14.16)
 - mkdir 292 (time 1788294639.55 total 20.08 last 14.93)
 - mkdir 434 (time 1788294649.62 total 30.16 last 14.09)
 - mkdir 569 (time 1788294659.62 total 40.16 last 13.50)
 - mkdir 714 (time 1788294669.66 total 50.19 last 14.45)
 - mkdir 863 (time 1788294679.78 total 60.31 last 14.72)
 - mkdir 984 (time 1788294689.83 total 70.36 last 12.04)
total: 1000 mkdir in 72.76 seconds: 13.74 ops/second
 - mkdir 144 (time 1788294629.33 total 10.00 last 14.40)
 - mkdir 292 (time 1788294639.36 total 20.03 last 14.75)
 - mkdir 429 (time 1788294649.39 total 30.06 last 13.67)
 - mkdir 557 (time 1788294659.41 total 40.08 last 12.77)
 - mkdir 704 (time 1788294669.44 total 50.11 last 14.66)
 - mkdir 843 (time 1788294679.44 total 60.11 last 13.90)
 - mkdir 970 (time 1788294689.49 total 70.17 last 12.63)
total: 1000 mkdir in 74.15 seconds: 13.49 ops/second
 - mkdir 127 (time 1788294628.72 total 10.10 last 12.57)
 - mkdir 274 (time 1788294638.77 total 20.16 last 14.62)
 - mkdir 414 (time 1788294648.79 total 30.18 last 13.97)
 - mkdir 542 (time 1788294658.81 total 40.20 last 12.77)
 - mkdir 678 (time 1788294668.83 total 50.22 last 13.57)
 - mkdir 816 (time 1788294678.88 total 60.27 last 13.74)
 - mkdir 943 (time 1788294689.00 total 70.39 last 12.54)
total: 1000 mkdir in 76.75 seconds: 13.03 ops/second
Time for file creation 87 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788294704 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1788294705 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1788294706 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1788294706 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1788294707 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1788294707 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1788294706 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1788294707 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1788294707 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1788294707 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
Time for file removal 54 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8