== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 13:56:58 (1789149418)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.65 seconds: 15.30 ops/second
 - mkdir 153 (time 1789149440.00 total 10.01 last 15.29)
 - mkdir 327 (time 1789149450.00 total 20.01 last 17.40)
 - mkdir 505 (time 1789149460.11 total 30.12 last 17.61)
 - mkdir 658 (time 1789149470.15 total 40.16 last 15.24)
 - mkdir 796 (time 1789149480.20 total 50.21 last 13.72)
 - mkdir 938 (time 1789149490.23 total 60.24 last 14.16)
total: 1000 mkdir in 63.96 seconds: 15.63 ops/second
 - mkdir 141 (time 1789149438.42 total 10.03 last 14.06)
 - mkdir 300 (time 1789149448.47 total 20.08 last 15.82)
 - mkdir 475 (time 1789149458.51 total 30.12 last 17.44)
 - mkdir 622 (time 1789149468.53 total 40.14 last 14.66)
 - mkdir 766 (time 1789149478.63 total 50.23 last 14.27)
 - mkdir 899 (time 1789149488.63 total 60.24 last 13.29)
total: 1000 mkdir in 67.36 seconds: 14.85 ops/second
 - mkdir 153 (time 1789149440.73 total 10.05 last 15.23)
 - mkdir 331 (time 1789149450.78 total 20.10 last 17.71)
 - mkdir 503 (time 1789149460.87 total 30.19 last 17.05)
 - mkdir 657 (time 1789149470.88 total 40.19 last 15.39)
 - mkdir 791 (time 1789149480.93 total 50.24 last 13.33)
 - mkdir 934 (time 1789149490.97 total 60.29 last 14.24)
total: 1000 mkdir in 65.14 seconds: 15.35 ops/second
 - mkdir 151 (time 1789149440.75 total 10.01 last 15.08)
 - mkdir 324 (time 1789149450.80 total 20.07 last 17.20)
 - mkdir 498 (time 1789149460.85 total 30.11 last 17.33)
 - mkdir 652 (time 1789149470.85 total 40.11 last 15.40)
 - mkdir 788 (time 1789149480.89 total 50.15 last 13.55)
 - mkdir 928 (time 1789149490.91 total 60.17 last 13.97)
total: 1000 mkdir in 65.75 seconds: 15.21 ops/second
 - mkdir 151 (time 1789149440.29 total 10.01 last 15.08)
 - mkdir 325 (time 1789149450.36 total 20.09 last 17.27)
 - mkdir 489 (time 1789149460.36 total 30.09 last 16.40)
 - mkdir 642 (time 1789149470.39 total 40.12 last 15.25)
 - mkdir 778 (time 1789149480.48 total 50.21 last 13.48)
 - mkdir 922 (time 1789149490.52 total 60.24 last 14.35)
total: 1000 mkdir in 66.28 seconds: 15.09 ops/second
 - mkdir 157 (time 1789149441.62 total 10.02 last 15.67)
 - mkdir 329 (time 1789149451.62 total 20.03 last 17.19)
 - mkdir 503 (time 1789149461.63 total 30.04 last 17.39)
 - mkdir 654 (time 1789149471.66 total 40.07 last 15.05)
 - mkdir 795 (time 1789149481.73 total 50.14 last 14.01)
 - mkdir 937 (time 1789149491.76 total 60.16 last 14.16)
total: 1000 mkdir in 65.21 seconds: 15.34 ops/second
 - mkdir 162 (time 1789149441.52 total 10.03 last 16.16)
 - mkdir 338 (time 1789149451.54 total 20.05 last 17.57)
 - mkdir 509 (time 1789149461.55 total 30.06 last 17.07)
 - mkdir 661 (time 1789149471.60 total 40.11 last 15.14)
 - mkdir 798 (time 1789149481.61 total 50.12 last 13.67)
 - mkdir 936 (time 1789149491.62 total 60.13 last 13.79)
total: 1000 mkdir in 65.33 seconds: 15.31 ops/second
 - mkdir 158 (time 1789149441.57 total 10.01 last 15.78)
 - mkdir 334 (time 1789149451.60 total 20.04 last 17.55)
 - mkdir 507 (time 1789149461.64 total 30.08 last 17.23)
 - mkdir 657 (time 1789149471.64 total 40.08 last 14.99)
 - mkdir 793 (time 1789149481.71 total 50.16 last 13.50)
 - mkdir 934 (time 1789149491.75 total 60.19 last 14.05)
total: 1000 mkdir in 65.64 seconds: 15.23 ops/second
 - mkdir 154 (time 1789149440.93 total 10.03 last 15.35)
 - mkdir 324 (time 1789149450.94 total 20.04 last 16.98)
 - mkdir 496 (time 1789149460.97 total 30.07 last 17.16)
 - mkdir 649 (time 1789149470.98 total 40.08 last 15.28)
 - mkdir 785 (time 1789149481.00 total 50.10 last 13.56)
 - mkdir 925 (time 1789149491.04 total 60.15 last 13.94)
total: 1000 mkdir in 66.40 seconds: 15.06 ops/second
 - mkdir 143 (time 1789149441.20 total 10.04 last 14.24)
 - mkdir 307 (time 1789149451.26 total 20.10 last 16.30)
 - mkdir 476 (time 1789149461.28 total 30.11 last 16.87)
 - mkdir 626 (time 1789149471.36 total 40.20 last 14.88)
 - mkdir 764 (time 1789149481.37 total 50.21 last 13.79)
 - mkdir 910 (time 1789149491.39 total 60.22 last 14.58)
total: 1000 mkdir in 67.90 seconds: 14.73 ops/second
Time for file creation 80 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 14 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.80 seconds: 12.54 ops/second
 - mkdir 158 (time 1789150115.02 total 10.09 last 15.65)
 - mkdir 311 (time 1789150125.03 total 20.11 last 15.28)
 - mkdir 448 (time 1789150135.06 total 30.14 last 13.65)
 - mkdir 584 (time 1789150145.09 total 40.17 last 13.56)
 - mkdir 738 (time 1789150155.14 total 50.22 last 15.32)
 - mkdir 876 (time 1789150165.17 total 60.24 last 13.76)
total: 1000 mkdir in 69.52 seconds: 14.38 ops/second
 - mkdir 141 (time 1789150111.74 total 10.01 last 14.09)
 - mkdir 286 (time 1789150121.75 total 20.02 last 14.48)
 - mkdir 422 (time 1789150131.89 total 30.16 last 13.42)
 - mkdir 552 (time 1789150142.03 total 40.30 last 12.83)
 - mkdir 708 (time 1789150152.06 total 50.32 last 15.55)
 - mkdir 845 (time 1789150162.12 total 60.39 last 13.62)
 - mkdir 970 (time 1789150172.15 total 70.42 last 12.46)
total: 1000 mkdir in 73.37 seconds: 13.63 ops/second
 - mkdir 156 (time 1789150113.42 total 10.04 last 15.54)
 - mkdir 294 (time 1789150123.44 total 20.05 last 13.78)
 - mkdir 428 (time 1789150133.48 total 30.09 last 13.35)
 - mkdir 556 (time 1789150143.48 total 40.10 last 12.79)
 - mkdir 712 (time 1789150153.55 total 50.16 last 15.50)
 - mkdir 845 (time 1789150163.66 total 60.27 last 13.16)
 - mkdir 978 (time 1789150173.68 total 70.30 last 13.26)
total: 1000 mkdir in 72.83 seconds: 13.73 ops/second
 - mkdir 158 (time 1789150114.40 total 10.05 last 15.73)
 - mkdir 302 (time 1789150124.49 total 20.14 last 14.27)
 - mkdir 430 (time 1789150134.58 total 30.23 last 12.69)
 - mkdir 562 (time 1789150144.60 total 40.25 last 13.17)
 - mkdir 714 (time 1789150154.62 total 50.27 last 15.17)
 - mkdir 845 (time 1789150164.67 total 60.31 last 13.04)
 - mkdir 980 (time 1789150174.75 total 70.40 last 13.39)
total: 1000 mkdir in 73.20 seconds: 13.66 ops/second
 - mkdir 155 (time 1789150114.62 total 10.23 last 15.16)
 - mkdir 298 (time 1789150124.62 total 20.23 last 14.30)
 - mkdir 427 (time 1789150134.63 total 30.23 last 12.89)
 - mkdir 558 (time 1789150144.64 total 40.25 last 13.08)
 - mkdir 710 (time 1789150154.72 total 50.32 last 15.09)
 - mkdir 845 (time 1789150164.75 total 60.36 last 13.45)
 - mkdir 978 (time 1789150174.79 total 70.40 last 13.25)
total: 1000 mkdir in 73.36 seconds: 13.63 ops/second
 - mkdir 152 (time 1789150113.18 total 10.01 last 15.18)
 - mkdir 293 (time 1789150123.24 total 20.08 last 14.01)
 - mkdir 424 (time 1789150133.25 total 30.08 last 13.09)
 - mkdir 550 (time 1789150143.31 total 40.14 last 12.53)
 - mkdir 709 (time 1789150153.35 total 50.19 last 15.83)
 - mkdir 839 (time 1789150163.43 total 60.26 last 12.90)
 - mkdir 965 (time 1789150173.46 total 70.30 last 12.55)
total: 1000 mkdir in 74.66 seconds: 13.39 ops/second
 - mkdir 156 (time 1789150114.21 total 10.04 last 15.54)
 - mkdir 297 (time 1789150124.25 total 20.08 last 14.04)
 - mkdir 431 (time 1789150134.26 total 30.09 last 13.39)
 - mkdir 558 (time 1789150144.27 total 40.10 last 12.69)
 - mkdir 713 (time 1789150154.29 total 50.12 last 15.47)
 - mkdir 846 (time 1789150164.34 total 60.17 last 13.23)
 - mkdir 975 (time 1789150174.44 total 70.28 last 12.76)
total: 1000 mkdir in 73.97 seconds: 13.52 ops/second
 - mkdir 149 (time 1789150114.15 total 10.04 last 14.84)
 - mkdir 292 (time 1789150124.20 total 20.09 last 14.24)
 - mkdir 418 (time 1789150134.22 total 30.11 last 12.56)
 - mkdir 546 (time 1789150144.23 total 40.12 last 12.79)
 - mkdir 706 (time 1789150154.31 total 50.20 last 15.87)
 - mkdir 841 (time 1789150164.35 total 60.24 last 13.45)
 - mkdir 971 (time 1789150174.38 total 70.27 last 12.96)
total: 1000 mkdir in 74.21 seconds: 13.47 ops/second
 - mkdir 154 (time 1789150114.38 total 10.08 last 15.27)
 - mkdir 300 (time 1789150124.40 total 20.10 last 14.58)
 - mkdir 428 (time 1789150134.42 total 30.12 last 12.78)
 - mkdir 562 (time 1789150144.43 total 40.13 last 13.39)
 - mkdir 709 (time 1789150154.50 total 50.20 last 14.59)
 - mkdir 841 (time 1789150164.56 total 60.26 last 13.12)
 - mkdir 972 (time 1789150174.62 total 70.32 last 13.03)
total: 1000 mkdir in 74.05 seconds: 13.50 ops/second
 - mkdir 148 (time 1789150114.86 total 10.04 last 14.74)
 - mkdir 292 (time 1789150124.88 total 20.06 last 14.37)
 - mkdir 424 (time 1789150134.95 total 30.14 last 13.10)
 - mkdir 558 (time 1789150144.96 total 40.14 last 13.39)
 - mkdir 707 (time 1789150154.98 total 50.16 last 14.87)
 - mkdir 840 (time 1789150165.03 total 60.21 last 13.24)
 - mkdir 962 (time 1789150175.14 total 70.32 last 12.06)
total: 1000 mkdir in 74.76 seconds: 13.38 ops/second
Time for file creation 87 sec for 1 RPC sent at a time
 - unlinked 0 (time 1789150188 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1789150190 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1789150191 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1789150191 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1789150191 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1789150191 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1789150191 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1789150191 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1789150191 ; total 0 ; last 0)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1789150191 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
Time for file removal 56 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8