== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 15:35:56 (1787859356)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.58 seconds: 17.31 ops/second
 - mkdir 159 (time 1787859377.84 total 10.05 last 15.82)
 - mkdir 369 (time 1787859387.91 total 20.12 last 20.87)
 - mkdir 584 (time 1787859397.94 total 30.15 last 21.43)
 - mkdir 779 (time 1787859407.99 total 40.20 last 19.39)
 - mkdir 939 (time 1787859418.11 total 50.32 last 15.81)
total: 1000 mkdir in 54.76 seconds: 18.26 ops/second
 - mkdir 181 (time 1787859378.88 total 10.03 last 18.05)
 - mkdir 385 (time 1787859388.90 total 20.04 last 20.37)
 - mkdir 597 (time 1787859398.93 total 30.08 last 21.13)
 - mkdir 781 (time 1787859408.95 total 40.10 last 18.36)
 - mkdir 942 (time 1787859419.02 total 50.17 last 15.99)
total: 1000 mkdir in 54.23 seconds: 18.44 ops/second
 - mkdir 166 (time 1787859378.43 total 10.03 last 16.56)
 - mkdir 369 (time 1787859388.45 total 20.05 last 20.25)
 - mkdir 581 (time 1787859398.47 total 30.06 last 21.18)
 - mkdir 772 (time 1787859408.50 total 40.09 last 19.04)
 - mkdir 930 (time 1787859418.50 total 50.09 last 15.80)
total: 1000 mkdir in 55.49 seconds: 18.02 ops/second
 - mkdir 181 (time 1787859379.17 total 10.01 last 18.08)
 - mkdir 388 (time 1787859389.20 total 20.04 last 20.64)
 - mkdir 595 (time 1787859399.24 total 30.08 last 20.62)
 - mkdir 784 (time 1787859409.27 total 40.10 last 18.85)
 - mkdir 934 (time 1787859419.35 total 50.18 last 14.89)
total: 1000 mkdir in 55.25 seconds: 18.10 ops/second
 - mkdir 169 (time 1787859378.01 total 10.05 last 16.82)
 - mkdir 366 (time 1787859388.02 total 20.06 last 19.67)
 - mkdir 565 (time 1787859398.05 total 30.10 last 19.83)
 - mkdir 758 (time 1787859408.10 total 40.14 last 19.22)
 - mkdir 918 (time 1787859418.12 total 50.16 last 15.97)
total: 1000 mkdir in 56.86 seconds: 17.59 ops/second
 - mkdir 181 (time 1787859379.32 total 10.02 last 18.06)
 - mkdir 391 (time 1787859389.35 total 20.05 last 20.95)
 - mkdir 592 (time 1787859399.37 total 30.07 last 20.05)
 - mkdir 779 (time 1787859409.38 total 40.08 last 18.68)
 - mkdir 930 (time 1787859419.41 total 50.11 last 15.06)
total: 1000 mkdir in 55.90 seconds: 17.89 ops/second
 - mkdir 165 (time 1787859377.94 total 10.03 last 16.44)
 - mkdir 368 (time 1787859387.99 total 20.08 last 20.20)
 - mkdir 567 (time 1787859398.01 total 30.10 last 19.87)
 - mkdir 756 (time 1787859408.01 total 40.10 last 18.89)
 - mkdir 917 (time 1787859418.07 total 50.16 last 16.00)
total: 1000 mkdir in 57.34 seconds: 17.44 ops/second
 - mkdir 178 (time 1787859379.15 total 10.00 last 17.79)
 - mkdir 387 (time 1787859389.16 total 20.01 last 20.88)
 - mkdir 594 (time 1787859399.17 total 30.02 last 20.69)
 - mkdir 781 (time 1787859409.22 total 40.07 last 18.61)
 - mkdir 932 (time 1787859419.31 total 50.16 last 14.95)
total: 1000 mkdir in 56.23 seconds: 17.78 ops/second
 - mkdir 183 (time 1787859379.39 total 10.05 last 18.22)
 - mkdir 381 (time 1787859389.39 total 20.05 last 19.80)
 - mkdir 589 (time 1787859399.40 total 30.05 last 20.79)
 - mkdir 776 (time 1787859409.47 total 40.12 last 18.58)
 - mkdir 926 (time 1787859419.53 total 50.18 last 14.91)
total: 1000 mkdir in 56.39 seconds: 17.73 ops/second
 - mkdir 167 (time 1787859379.29 total 10.03 last 16.65)
 - mkdir 357 (time 1787859389.33 total 20.06 last 18.94)
 - mkdir 552 (time 1787859399.37 total 30.10 last 19.42)
 - mkdir 739 (time 1787859409.37 total 40.11 last 18.69)
 - mkdir 886 (time 1787859419.43 total 50.16 last 14.62)
total: 1000 mkdir in 59.33 seconds: 16.85 ops/second
Time for file creation 70 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.76 seconds: 13.12 ops/second
 - mkdir 114 (time 1787859995.42 total 10.04 last 11.36)
 - mkdir 257 (time 1787860005.44 total 20.06 last 14.27)
 - mkdir 416 (time 1787860015.47 total 30.09 last 15.85)
 - mkdir 560 (time 1787860025.49 total 40.11 last 14.37)
 - mkdir 702 (time 1787860035.60 total 50.22 last 14.06)
 - mkdir 867 (time 1787860045.61 total 60.23 last 16.48)
total: 1000 mkdir in 67.82 seconds: 14.75 ops/second
 - mkdir 116 (time 1787859994.95 total 10.04 last 11.55)
 - mkdir 251 (time 1787860004.97 total 20.06 last 13.48)
 - mkdir 405 (time 1787860015.08 total 30.16 last 15.24)
 - mkdir 540 (time 1787860025.10 total 40.19 last 13.47)
 - mkdir 673 (time 1787860035.14 total 50.22 last 13.25)
 - mkdir 840 (time 1787860045.18 total 60.26 last 16.63)
 - mkdir 996 (time 1787860055.23 total 70.31 last 15.53)
total: 1000 mkdir in 70.62 seconds: 14.16 ops/second
 - mkdir 119 (time 1787859995.60 total 10.09 last 11.79)
 - mkdir 255 (time 1787860005.65 total 20.14 last 13.53)
 - mkdir 407 (time 1787860015.71 total 30.21 last 15.10)
 - mkdir 547 (time 1787860025.83 total 40.33 last 13.84)
 - mkdir 684 (time 1787860035.84 total 50.33 last 13.69)
 - mkdir 848 (time 1787860045.90 total 60.39 last 16.30)
 - mkdir 998 (time 1787860055.92 total 70.41 last 14.97)
total: 1000 mkdir in 70.62 seconds: 14.16 ops/second
 - mkdir 103 (time 1787859993.26 total 10.10 last 10.20)
 - mkdir 243 (time 1787860003.31 total 20.15 last 13.93)
 - mkdir 389 (time 1787860013.43 total 30.26 last 14.44)
 - mkdir 532 (time 1787860023.48 total 40.32 last 14.23)
 - mkdir 654 (time 1787860033.56 total 50.39 last 12.10)
 - mkdir 807 (time 1787860043.57 total 60.41 last 15.27)
 - mkdir 972 (time 1787860053.60 total 70.44 last 16.46)
total: 1000 mkdir in 73.64 seconds: 13.58 ops/second
 - mkdir 105 (time 1787859994.61 total 10.03 last 10.47)
 - mkdir 236 (time 1787860004.63 total 20.04 last 13.09)
 - mkdir 387 (time 1787860014.66 total 30.08 last 15.04)
 - mkdir 523 (time 1787860024.71 total 40.12 last 13.54)
 - mkdir 654 (time 1787860034.72 total 50.13 last 13.08)
 - mkdir 812 (time 1787860044.72 total 60.14 last 15.79)
 - mkdir 972 (time 1787860054.73 total 70.15 last 15.98)
total: 1000 mkdir in 73.54 seconds: 13.60 ops/second
 - mkdir 101 (time 1787859994.08 total 10.03 last 10.07)
 - mkdir 232 (time 1787860004.10 total 20.05 last 13.08)
 - mkdir 379 (time 1787860014.11 total 30.05 last 14.69)
 - mkdir 521 (time 1787860024.15 total 40.10 last 14.14)
 - mkdir 650 (time 1787860034.21 total 50.15 last 12.83)
 - mkdir 803 (time 1787860044.29 total 60.24 last 15.17)
 - mkdir 966 (time 1787860054.41 total 70.36 last 16.11)
total: 1000 mkdir in 74.42 seconds: 13.44 ops/second
 - mkdir 99 (time 1787859993.73 total 10.03 last 9.87)
 - mkdir 235 (time 1787860003.76 total 20.06 last 13.56)
 - mkdir 387 (time 1787860013.77 total 30.07 last 15.19)
 - mkdir 522 (time 1787860023.81 total 40.11 last 13.45)
 - mkdir 652 (time 1787860033.83 total 50.13 last 12.98)
 - mkdir 799 (time 1787860043.85 total 60.15 last 14.67)
 - mkdir 962 (time 1787860053.96 total 70.26 last 16.12)
total: 1000 mkdir in 75.37 seconds: 13.27 ops/second
 - mkdir 120 (time 1787859995.83 total 10.05 last 11.94)
 - mkdir 253 (time 1787860005.89 total 20.11 last 13.23)
 - mkdir 403 (time 1787860015.96 total 30.18 last 14.90)
 - mkdir 532 (time 1787860025.96 total 40.18 last 12.89)
 - mkdir 669 (time 1787860035.99 total 50.21 last 13.66)
 - mkdir 823 (time 1787860046.01 total 60.23 last 15.37)
 - mkdir 977 (time 1787860056.08 total 70.30 last 15.29)
total: 1000 mkdir in 73.37 seconds: 13.63 ops/second
 - mkdir 88 (time 1787859991.13 total 10.03 last 8.77)
 - mkdir 225 (time 1787860001.28 total 20.17 last 13.51)
 - mkdir 350 (time 1787860011.32 total 30.22 last 12.44)
 - mkdir 489 (time 1787860021.35 total 40.24 last 13.87)
 - mkdir 608 (time 1787860031.35 total 50.25 last 11.89)
 - mkdir 749 (time 1787860041.36 total 60.25 last 14.09)
 - mkdir 910 (time 1787860051.45 total 70.35 last 15.95)
total: 1000 mkdir in 78.61 seconds: 12.72 ops/second
 - mkdir 106 (time 1787859994.78 total 10.05 last 10.55)
 - mkdir 237 (time 1787860004.82 total 20.09 last 13.06)
 - mkdir 386 (time 1787860014.82 total 30.09 last 14.90)
 - mkdir 520 (time 1787860024.93 total 40.20 last 13.26)
 - mkdir 643 (time 1787860034.95 total 50.22 last 12.27)
 - mkdir 794 (time 1787860044.98 total 60.25 last 15.06)
 - mkdir 944 (time 1787860055.02 total 70.29 last 14.93)
total: 1000 mkdir in 76.17 seconds: 13.13 ops/second
Time for file creation 89 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787860070 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787860070 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787860070 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787860070 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787860071 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787860071 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787860071 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787860071 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787860071 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787860071 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
Time for file removal 47 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8