== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 07:21:15 (1789039275)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.57 seconds: 17.48 ops/second
 - mkdir 126 (time 1789039301.25 total 10.08 last 12.50)
 - mkdir 280 (time 1789039311.32 total 20.15 last 15.30)
 - mkdir 449 (time 1789039321.32 total 30.15 last 16.90)
 - mkdir 583 (time 1789039331.37 total 40.19 last 13.34)
 - mkdir 734 (time 1789039341.39 total 50.22 last 15.06)
 - mkdir 859 (time 1789039351.42 total 60.25 last 12.46)
 - mkdir 991 (time 1789039361.45 total 70.27 last 13.17)
total: 1000 mkdir in 70.94 seconds: 14.10 ops/second
 - mkdir 124 (time 1789039299.23 total 10.02 last 12.37)
 - mkdir 257 (time 1789039309.25 total 20.05 last 13.27)
 - mkdir 424 (time 1789039319.27 total 30.07 last 16.66)
 - mkdir 571 (time 1789039329.33 total 40.13 last 14.61)
 - mkdir 712 (time 1789039339.38 total 50.17 last 14.04)
 - mkdir 843 (time 1789039349.39 total 60.18 last 13.09)
 - mkdir 965 (time 1789039359.41 total 70.21 last 12.17)
total: 1000 mkdir in 72.92 seconds: 13.71 ops/second
 - mkdir 119 (time 1789039300.97 total 10.05 last 11.85)
 - mkdir 268 (time 1789039311.01 total 20.08 last 14.84)
 - mkdir 435 (time 1789039321.09 total 30.16 last 16.58)
 - mkdir 572 (time 1789039331.11 total 40.18 last 13.67)
 - mkdir 723 (time 1789039341.12 total 50.19 last 15.09)
 - mkdir 848 (time 1789039351.17 total 60.24 last 12.43)
 - mkdir 976 (time 1789039361.19 total 70.26 last 12.77)
total: 1000 mkdir in 73.27 seconds: 13.65 ops/second
 - mkdir 117 (time 1789039301.52 total 10.05 last 11.65)
 - mkdir 267 (time 1789039311.59 total 20.12 last 14.89)
 - mkdir 436 (time 1789039321.64 total 30.18 last 16.81)
 - mkdir 572 (time 1789039331.72 total 40.25 last 13.50)
 - mkdir 726 (time 1789039341.77 total 50.30 last 15.33)
 - mkdir 852 (time 1789039351.83 total 60.36 last 12.52)
 - mkdir 983 (time 1789039361.86 total 70.39 last 13.06)
total: 1000 mkdir in 72.87 seconds: 13.72 ops/second
 - mkdir 115 (time 1789039300.18 total 10.06 last 11.43)
 - mkdir 264 (time 1789039310.21 total 20.10 last 14.85)
 - mkdir 421 (time 1789039320.31 total 30.20 last 15.54)
 - mkdir 562 (time 1789039330.42 total 40.30 last 13.95)
 - mkdir 710 (time 1789039340.44 total 50.32 last 14.77)
 - mkdir 833 (time 1789039350.47 total 60.36 last 12.26)
 - mkdir 964 (time 1789039360.54 total 70.42 last 13.01)
total: 1000 mkdir in 75.41 seconds: 13.26 ops/second
 - mkdir 122 (time 1789039301.54 total 10.11 last 12.07)
 - mkdir 278 (time 1789039311.59 total 20.16 last 15.52)
 - mkdir 439 (time 1789039321.66 total 30.23 last 15.99)
 - mkdir 571 (time 1789039331.67 total 40.24 last 13.18)
 - mkdir 724 (time 1789039341.68 total 50.25 last 15.28)
 - mkdir 849 (time 1789039351.76 total 60.33 last 12.40)
 - mkdir 975 (time 1789039361.86 total 70.42 last 12.48)
total: 1000 mkdir in 74.40 seconds: 13.44 ops/second
 - mkdir 116 (time 1789039301.84 total 10.04 last 11.56)
 - mkdir 271 (time 1789039311.89 total 20.09 last 15.41)
 - mkdir 438 (time 1789039321.95 total 30.15 last 16.61)
 - mkdir 569 (time 1789039332.01 total 40.21 last 13.02)
 - mkdir 719 (time 1789039342.04 total 50.25 last 14.95)
 - mkdir 845 (time 1789039352.05 total 60.25 last 12.60)
 - mkdir 972 (time 1789039362.11 total 70.31 last 12.62)
total: 1000 mkdir in 74.99 seconds: 13.33 ops/second
 - mkdir 122 (time 1789039301.96 total 10.08 last 12.10)
 - mkdir 276 (time 1789039311.96 total 20.08 last 15.39)
 - mkdir 439 (time 1789039322.00 total 30.12 last 16.24)
 - mkdir 575 (time 1789039332.08 total 40.20 last 13.50)
 - mkdir 723 (time 1789039342.20 total 50.32 last 14.62)
 - mkdir 844 (time 1789039352.22 total 60.34 last 12.08)
 - mkdir 965 (time 1789039362.28 total 70.40 last 12.03)
total: 1000 mkdir in 75.59 seconds: 13.23 ops/second
 - mkdir 114 (time 1789039302.15 total 10.04 last 11.35)
 - mkdir 271 (time 1789039312.20 total 20.09 last 15.63)
 - mkdir 424 (time 1789039322.21 total 30.10 last 15.28)
 - mkdir 557 (time 1789039332.21 total 40.10 last 13.30)
 - mkdir 710 (time 1789039342.21 total 50.10 last 15.30)
 - mkdir 830 (time 1789039352.22 total 60.11 last 11.99)
 - mkdir 954 (time 1789039362.26 total 70.15 last 12.35)
total: 1000 mkdir in 76.94 seconds: 13.00 ops/second
 - mkdir 111 (time 1789039301.99 total 10.05 last 11.05)
 - mkdir 260 (time 1789039312.02 total 20.08 last 14.85)
 - mkdir 413 (time 1789039322.04 total 30.10 last 15.27)
 - mkdir 549 (time 1789039332.11 total 40.18 last 13.50)
 - mkdir 700 (time 1789039342.14 total 50.21 last 15.06)
 - mkdir 822 (time 1789039352.15 total 60.21 last 12.19)
 - mkdir 945 (time 1789039362.29 total 70.36 last 12.13)
total: 1000 mkdir in 78.61 seconds: 12.72 ops/second
Time for file creation 92 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 14 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.52 seconds: 19.17 ops/second
 - mkdir 134 (time 1789039952.04 total 10.13 last 13.23)
 - mkdir 256 (time 1789039962.12 total 20.21 last 12.11)
 - mkdir 397 (time 1789039972.14 total 30.23 last 14.06)
 - mkdir 527 (time 1789039982.15 total 40.24 last 12.99)
 - mkdir 661 (time 1789039992.17 total 50.26 last 13.37)
 - mkdir 792 (time 1789040002.22 total 60.31 last 13.03)
 - mkdir 922 (time 1789040012.26 total 70.35 last 12.95)
total: 1000 mkdir in 75.72 seconds: 13.21 ops/second
 - mkdir 130 (time 1789039951.41 total 10.06 last 12.92)
 - mkdir 249 (time 1789039961.42 total 20.07 last 11.89)
 - mkdir 380 (time 1789039971.47 total 30.12 last 13.04)
 - mkdir 512 (time 1789039981.52 total 40.17 last 13.14)
 - mkdir 644 (time 1789039991.53 total 50.18 last 13.18)
 - mkdir 775 (time 1789040001.62 total 60.27 last 12.98)
 - mkdir 909 (time 1789040011.64 total 70.29 last 13.38)
total: 1000 mkdir in 77.53 seconds: 12.90 ops/second
 - mkdir 123 (time 1789039950.13 total 10.06 last 12.22)
 - mkdir 241 (time 1789039960.15 total 20.09 last 11.77)
 - mkdir 367 (time 1789039970.22 total 30.16 last 12.51)
 - mkdir 499 (time 1789039980.35 total 40.28 last 13.04)
 - mkdir 634 (time 1789039990.37 total 50.30 last 13.48)
 - mkdir 762 (time 1789040000.38 total 60.32 last 12.78)
 - mkdir 891 (time 1789040010.39 total 70.32 last 12.89)
total: 1000 mkdir in 79.24 seconds: 12.62 ops/second
 - mkdir 128 (time 1789039951.58 total 10.01 last 12.79)
 - mkdir 247 (time 1789039961.61 total 20.03 last 11.87)
 - mkdir 380 (time 1789039971.63 total 30.06 last 13.27)
 - mkdir 516 (time 1789039981.69 total 40.11 last 13.52)
 - mkdir 648 (time 1789039991.77 total 50.20 last 13.09)
 - mkdir 779 (time 1789040001.82 total 60.25 last 13.03)
 - mkdir 906 (time 1789040011.83 total 70.25 last 12.69)
total: 1000 mkdir in 77.82 seconds: 12.85 ops/second
 - mkdir 131 (time 1789039952.17 total 10.09 last 12.98)
 - mkdir 255 (time 1789039962.21 total 20.12 last 12.36)
 - mkdir 390 (time 1789039972.29 total 30.21 last 13.39)
 - mkdir 522 (time 1789039982.34 total 40.25 last 13.14)
 - mkdir 652 (time 1789039992.36 total 50.28 last 12.97)
 - mkdir 780 (time 1789040002.36 total 60.28 last 12.79)
 - mkdir 905 (time 1789040012.48 total 70.40 last 12.36)
total: 1000 mkdir in 77.99 seconds: 12.82 ops/second
 - mkdir 131 (time 1789039952.55 total 10.02 last 13.07)
 - mkdir 254 (time 1789039962.60 total 20.08 last 12.23)
 - mkdir 384 (time 1789039972.65 total 30.13 last 12.94)
 - mkdir 516 (time 1789039982.71 total 40.19 last 13.12)
 - mkdir 649 (time 1789039992.73 total 50.21 last 13.28)
 - mkdir 780 (time 1789040002.79 total 60.27 last 13.02)
 - mkdir 909 (time 1789040012.85 total 70.33 last 12.83)
total: 1000 mkdir in 78.03 seconds: 12.82 ops/second
 - mkdir 127 (time 1789039951.84 total 10.03 last 12.66)
 - mkdir 246 (time 1789039961.94 total 20.12 last 11.79)
 - mkdir 375 (time 1789039971.94 total 30.13 last 12.90)
 - mkdir 504 (time 1789039982.00 total 40.19 last 12.82)
 - mkdir 636 (time 1789039992.08 total 50.26 last 13.10)
 - mkdir 767 (time 1789040002.15 total 60.33 last 13.01)
 - mkdir 898 (time 1789040012.15 total 70.33 last 13.10)
total: 1000 mkdir in 79.41 seconds: 12.59 ops/second
 - mkdir 129 (time 1789039952.32 total 10.00 last 12.90)
 - mkdir 248 (time 1789039962.37 total 20.05 last 11.85)
 - mkdir 381 (time 1789039972.43 total 30.11 last 13.21)
 - mkdir 509 (time 1789039982.46 total 40.14 last 12.76)
 - mkdir 639 (time 1789039992.51 total 50.19 last 12.94)
 - mkdir 768 (time 1789040002.58 total 60.25 last 12.82)
 - mkdir 897 (time 1789040012.59 total 70.27 last 12.88)
total: 1000 mkdir in 79.70 seconds: 12.55 ops/second
 - mkdir 132 (time 1789039952.54 total 10.12 last 13.04)
 - mkdir 253 (time 1789039962.61 total 20.20 last 12.01)
 - mkdir 382 (time 1789039972.72 total 30.31 last 12.76)
 - mkdir 514 (time 1789039982.76 total 40.35 last 13.15)
 - mkdir 645 (time 1789039992.81 total 50.39 last 13.04)
 - mkdir 773 (time 1789040002.85 total 60.44 last 12.74)
 - mkdir 897 (time 1789040012.86 total 70.45 last 12.39)
total: 1000 mkdir in 79.86 seconds: 12.52 ops/second
 - mkdir 119 (time 1789039951.87 total 10.04 last 11.85)
 - mkdir 235 (time 1789039961.94 total 20.11 last 11.53)
 - mkdir 362 (time 1789039971.95 total 30.12 last 12.68)
 - mkdir 496 (time 1789039982.03 total 40.20 last 13.30)
 - mkdir 631 (time 1789039992.07 total 50.24 last 13.45)
 - mkdir 760 (time 1789040002.11 total 60.28 last 12.85)
 - mkdir 884 (time 1789040012.17 total 70.34 last 12.33)
 - mkdir 988 (time 1789040022.19 total 80.36 last 10.39)
total: 1000 mkdir in 81.60 seconds: 12.25 ops/second
Time for file creation 93 sec for 1 RPC sent at a time
 - unlinked 0 (time 1789040032 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1789040033 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1789040033 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1789040034 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1789040034 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1789040034 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1789040034 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1789040034 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1789040035 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1789040035 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
Time for file removal 49 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8