== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 08:05:05 (1787400305)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.66 seconds: 15.07 ops/second
 - mkdir 125 (time 1787400324.83 total 10.09 last 12.39)
 - mkdir 282 (time 1787400334.88 total 20.14 last 15.62)
 - mkdir 453 (time 1787400344.89 total 30.15 last 17.07)
 - mkdir 591 (time 1787400354.96 total 40.22 last 13.71)
 - mkdir 725 (time 1787400364.97 total 50.23 last 13.39)
 - mkdir 888 (time 1787400375.02 total 60.28 last 16.21)
total: 1000 mkdir in 67.90 seconds: 14.73 ops/second
 - mkdir 136 (time 1787400328.15 total 10.04 last 13.54)
 - mkdir 322 (time 1787400338.19 total 20.09 last 18.52)
 - mkdir 482 (time 1787400348.20 total 30.10 last 15.98)
 - mkdir 601 (time 1787400358.27 total 40.16 last 11.83)
 - mkdir 750 (time 1787400368.31 total 50.21 last 14.83)
 - mkdir 908 (time 1787400378.32 total 60.22 last 15.78)
total: 1000 mkdir in 67.49 seconds: 14.82 ops/second
 - mkdir 127 (time 1787400327.41 total 10.04 last 12.65)
 - mkdir 300 (time 1787400337.43 total 20.05 last 17.27)
 - mkdir 460 (time 1787400347.45 total 30.08 last 15.96)
 - mkdir 583 (time 1787400357.48 total 40.10 last 12.27)
 - mkdir 732 (time 1787400367.51 total 50.14 last 14.84)
 - mkdir 887 (time 1787400377.53 total 60.15 last 15.48)
total: 1000 mkdir in 69.76 seconds: 14.33 ops/second
 - mkdir 135 (time 1787400327.89 total 10.05 last 13.44)
 - mkdir 314 (time 1787400337.92 total 20.08 last 17.84)
 - mkdir 466 (time 1787400348.03 total 30.19 last 15.03)
 - mkdir 588 (time 1787400358.09 total 40.25 last 12.13)
 - mkdir 737 (time 1787400368.13 total 50.29 last 14.84)
 - mkdir 894 (time 1787400378.17 total 60.33 last 15.64)
total: 1000 mkdir in 69.32 seconds: 14.43 ops/second
 - mkdir 133 (time 1787400327.96 total 10.10 last 13.16)
 - mkdir 305 (time 1787400337.96 total 20.11 last 17.19)
 - mkdir 460 (time 1787400347.97 total 30.12 last 15.49)
 - mkdir 585 (time 1787400358.08 total 40.23 last 12.36)
 - mkdir 733 (time 1787400368.09 total 50.23 last 14.79)
 - mkdir 893 (time 1787400378.12 total 60.27 last 15.94)
total: 1000 mkdir in 69.89 seconds: 14.31 ops/second
 - mkdir 135 (time 1787400328.04 total 10.05 last 13.43)
 - mkdir 313 (time 1787400338.04 total 20.05 last 17.80)
 - mkdir 464 (time 1787400348.12 total 30.13 last 14.98)
 - mkdir 586 (time 1787400358.15 total 40.16 last 12.17)
 - mkdir 735 (time 1787400368.18 total 50.19 last 14.86)
 - mkdir 892 (time 1787400378.20 total 60.21 last 15.66)
total: 1000 mkdir in 69.97 seconds: 14.29 ops/second
 - mkdir 124 (time 1787400327.40 total 10.00 last 12.40)
 - mkdir 290 (time 1787400337.42 total 20.02 last 16.56)
 - mkdir 448 (time 1787400347.46 total 30.06 last 15.74)
 - mkdir 569 (time 1787400357.47 total 40.07 last 12.09)
 - mkdir 715 (time 1787400367.49 total 50.09 last 14.57)
 - mkdir 874 (time 1787400377.58 total 60.18 last 15.76)
 - mkdir 990 (time 1787400387.64 total 70.25 last 11.53)
total: 1000 mkdir in 71.37 seconds: 14.01 ops/second
 - mkdir 135 (time 1787400328.34 total 10.06 last 13.41)
 - mkdir 306 (time 1787400338.36 total 20.08 last 17.07)
 - mkdir 461 (time 1787400348.40 total 30.12 last 15.45)
 - mkdir 579 (time 1787400358.42 total 40.14 last 11.78)
 - mkdir 728 (time 1787400368.42 total 50.14 last 14.89)
 - mkdir 883 (time 1787400378.43 total 60.15 last 15.49)
 - mkdir 993 (time 1787400388.44 total 70.16 last 10.99)
total: 1000 mkdir in 70.77 seconds: 14.13 ops/second
 - mkdir 127 (time 1787400327.97 total 10.04 last 12.65)
 - mkdir 294 (time 1787400337.99 total 20.06 last 16.66)
 - mkdir 448 (time 1787400348.04 total 30.11 last 15.33)
 - mkdir 571 (time 1787400358.06 total 40.13 last 12.27)
 - mkdir 720 (time 1787400368.10 total 50.17 last 14.84)
 - mkdir 876 (time 1787400378.16 total 60.23 last 15.52)
 - mkdir 981 (time 1787400388.19 total 70.26 last 10.46)
total: 1000 mkdir in 72.56 seconds: 13.78 ops/second
 - mkdir 118 (time 1787400326.94 total 10.01 last 11.79)
 - mkdir 279 (time 1787400336.95 total 20.02 last 16.08)
 - mkdir 435 (time 1787400347.00 total 30.07 last 15.53)
 - mkdir 558 (time 1787400357.00 total 40.07 last 12.30)
 - mkdir 698 (time 1787400367.00 total 50.07 last 14.00)
 - mkdir 857 (time 1787400377.05 total 60.12 last 15.82)
 - mkdir 972 (time 1787400387.21 total 70.28 last 11.31)
total: 1000 mkdir in 74.04 seconds: 13.51 ops/second
Time for file creation 85 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.46 seconds: 21.96 ops/second
 - mkdir 164 (time 1787400979.75 total 10.03 last 16.34)
 - mkdir 357 (time 1787400989.78 total 20.06 last 19.25)
 - mkdir 549 (time 1787400999.79 total 30.08 last 19.17)
 - mkdir 744 (time 1787401009.83 total 40.11 last 19.43)
 - mkdir 903 (time 1787401019.84 total 50.13 last 15.88)
total: 1000 mkdir in 55.82 seconds: 17.92 ops/second
 - mkdir 163 (time 1787400979.76 total 10.01 last 16.28)
 - mkdir 356 (time 1787400989.78 total 20.03 last 19.26)
 - mkdir 543 (time 1787400999.78 total 30.03 last 18.70)
 - mkdir 737 (time 1787401009.80 total 40.05 last 19.36)
 - mkdir 894 (time 1787401019.82 total 50.07 last 15.67)
total: 1000 mkdir in 56.15 seconds: 17.81 ops/second
 - mkdir 167 (time 1787400979.86 total 10.05 last 16.62)
 - mkdir 360 (time 1787400989.93 total 20.12 last 19.17)
 - mkdir 552 (time 1787400999.94 total 30.13 last 19.19)
 - mkdir 745 (time 1787401009.98 total 40.16 last 19.23)
 - mkdir 903 (time 1787401019.98 total 50.16 last 15.80)
total: 1000 mkdir in 56.11 seconds: 17.82 ops/second
 - mkdir 169 (time 1787400981.18 total 10.00 last 16.89)
 - mkdir 368 (time 1787400991.22 total 20.05 last 19.81)
 - mkdir 571 (time 1787401001.23 total 30.05 last 20.28)
 - mkdir 752 (time 1787401011.25 total 40.07 last 18.07)
 - mkdir 916 (time 1787401021.26 total 50.09 last 16.37)
total: 1000 mkdir in 55.23 seconds: 18.11 ops/second
 - mkdir 171 (time 1787400981.34 total 10.03 last 17.04)
 - mkdir 372 (time 1787400991.36 total 20.05 last 20.06)
 - mkdir 559 (time 1787401001.39 total 30.08 last 18.65)
 - mkdir 740 (time 1787401011.45 total 40.14 last 17.99)
 - mkdir 903 (time 1787401021.47 total 50.16 last 16.28)
total: 1000 mkdir in 56.79 seconds: 17.61 ops/second
 - mkdir 172 (time 1787400981.50 total 10.07 last 17.09)
 - mkdir 370 (time 1787400991.50 total 20.07 last 19.79)
 - mkdir 560 (time 1787401001.53 total 30.10 last 18.94)
 - mkdir 741 (time 1787401011.61 total 40.18 last 17.96)
 - mkdir 905 (time 1787401021.63 total 50.20 last 16.37)
total: 1000 mkdir in 56.71 seconds: 17.63 ops/second
 - mkdir 161 (time 1787400980.35 total 10.02 last 16.07)
 - mkdir 348 (time 1787400990.35 total 20.03 last 18.69)
 - mkdir 537 (time 1787401000.44 total 30.11 last 18.74)
 - mkdir 718 (time 1787401010.47 total 40.14 last 18.04)
 - mkdir 881 (time 1787401020.50 total 50.17 last 16.26)
total: 1000 mkdir in 57.90 seconds: 17.27 ops/second
 - mkdir 167 (time 1787400981.32 total 10.03 last 16.64)
 - mkdir 358 (time 1787400991.32 total 20.04 last 19.09)
 - mkdir 549 (time 1787401001.33 total 30.05 last 19.09)
 - mkdir 734 (time 1787401011.39 total 40.11 last 18.39)
 - mkdir 896 (time 1787401021.42 total 50.14 last 16.15)
total: 1000 mkdir in 57.17 seconds: 17.49 ops/second
 - mkdir 160 (time 1787400980.79 total 10.09 last 15.87)
 - mkdir 348 (time 1787400990.79 total 20.09 last 18.78)
 - mkdir 534 (time 1787401000.84 total 30.14 last 18.51)
 - mkdir 714 (time 1787401010.89 total 40.19 last 17.91)
 - mkdir 880 (time 1787401020.90 total 50.20 last 16.59)
total: 1000 mkdir in 58.03 seconds: 17.23 ops/second
 - mkdir 173 (time 1787400981.67 total 10.05 last 17.22)
 - mkdir 365 (time 1787400991.68 total 20.06 last 19.17)
 - mkdir 554 (time 1787401001.69 total 30.07 last 18.88)
 - mkdir 731 (time 1787401011.72 total 40.10 last 17.66)
 - mkdir 890 (time 1787401021.77 total 50.15 last 15.81)
total: 1000 mkdir in 57.79 seconds: 17.30 ops/second
Time for file creation 68 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787401036 ; total 0 ; last 0)
total: 1000 unlinks in 23 seconds: 43.478260 unlinks/second
 - unlinked 0 (time 1787401036 ; total 0 ; last 0)
total: 1000 unlinks in 24 seconds: 41.666668 unlinks/second
 - unlinked 0 (time 1787401037 ; total 0 ; last 0)
total: 1000 unlinks in 24 seconds: 41.666668 unlinks/second
 - unlinked 0 (time 1787401037 ; total 0 ; last 0)
total: 1000 unlinks in 25 seconds: 40.000000 unlinks/second
 - unlinked 0 (time 1787401037 ; total 0 ; last 0)
total: 1000 unlinks in 25 seconds: 40.000000 unlinks/second
 - unlinked 0 (time 1787401037 ; total 0 ; last 0)
total: 1000 unlinks in 25 seconds: 40.000000 unlinks/second
 - unlinked 0 (time 1787401037 ; total 0 ; last 0)
total: 1000 unlinks in 25 seconds: 40.000000 unlinks/second
 - unlinked 0 (time 1787401037 ; total 0 ; last 0)
total: 1000 unlinks in 26 seconds: 38.461540 unlinks/second
 - unlinked 0 (time 1787401038 ; total 0 ; last 0)
total: 1000 unlinks in 25 seconds: 40.000000 unlinks/second
 - unlinked 0 (time 1787401038 ; total 0 ; last 0)
total: 1000 unlinks in 25 seconds: 40.000000 unlinks/second
Time for file removal 34 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8