== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 07:49:58 (1787485798)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.54 seconds: 18.67 ops/second
 - mkdir 112 (time 1787485823.51 total 10.02 last 11.18)
 - mkdir 222 (time 1787485833.59 total 20.10 last 10.91)
 - mkdir 294 (time 1787485843.62 total 30.14 last 7.17)
 - mkdir 460 (time 1787485853.71 total 40.23 last 16.45)
 - mkdir 578 (time 1787485863.80 total 50.32 last 11.69)
 - mkdir 671 (time 1787485873.84 total 60.36 last 9.26)
 - mkdir 796 (time 1787485883.87 total 70.38 last 12.47)
 - mkdir 949 (time 1787485893.88 total 80.40 last 15.27)
total: 1000 mkdir in 83.76 seconds: 11.94 ops/second
 - mkdir 112 (time 1787485823.48 total 10.02 last 11.18)
 - mkdir 222 (time 1787485833.59 total 20.13 last 10.88)
 - mkdir 294 (time 1787485843.67 total 30.21 last 7.14)
 - mkdir 458 (time 1787485853.67 total 40.21 last 16.39)
 - mkdir 575 (time 1787485863.69 total 50.24 last 11.67)
 - mkdir 667 (time 1787485873.78 total 60.32 last 9.12)
 - mkdir 784 (time 1787485883.79 total 70.33 last 11.69)
 - mkdir 940 (time 1787485893.85 total 80.39 last 15.51)
total: 1000 mkdir in 84.99 seconds: 11.77 ops/second
 - mkdir 104 (time 1787485824.25 total 10.08 last 10.32)
 - mkdir 210 (time 1787485834.25 total 20.08 last 10.60)
 - mkdir 290 (time 1787485844.28 total 30.10 last 7.98)
 - mkdir 454 (time 1787485854.32 total 40.14 last 16.34)
 - mkdir 579 (time 1787485864.42 total 50.24 last 12.38)
 - mkdir 669 (time 1787485874.44 total 60.26 last 8.98)
 - mkdir 800 (time 1787485884.48 total 70.31 last 13.05)
 - mkdir 952 (time 1787485894.50 total 80.32 last 15.17)
total: 1000 mkdir in 84.43 seconds: 11.84 ops/second
 - mkdir 102 (time 1787485821.90 total 10.09 last 10.11)
 - mkdir 215 (time 1787485831.99 total 20.18 last 11.20)
 - mkdir 277 (time 1787485842.17 total 30.36 last 6.09)
 - mkdir 441 (time 1787485852.20 total 40.39 last 16.35)
 - mkdir 561 (time 1787485862.24 total 50.42 last 11.96)
 - mkdir 654 (time 1787485872.28 total 60.46 last 9.26)
 - mkdir 754 (time 1787485882.30 total 70.48 last 9.99)
 - mkdir 906 (time 1787485892.33 total 80.52 last 15.15)
total: 1000 mkdir in 87.47 seconds: 11.43 ops/second
 - mkdir 109 (time 1787485823.54 total 10.15 last 10.74)
 - mkdir 218 (time 1787485833.55 total 20.16 last 10.89)
 - mkdir 288 (time 1787485843.58 total 30.18 last 6.98)
 - mkdir 452 (time 1787485853.64 total 40.25 last 16.29)
 - mkdir 575 (time 1787485863.83 total 50.44 last 12.07)
 - mkdir 665 (time 1787485873.97 total 60.58 last 8.88)
 - mkdir 788 (time 1787485884.00 total 70.60 last 12.27)
 - mkdir 941 (time 1787485894.05 total 80.66 last 15.22)
total: 1000 mkdir in 86.46 seconds: 11.57 ops/second
 - mkdir 108 (time 1787485824.50 total 10.07 last 10.72)
 - mkdir 221 (time 1787485834.56 total 20.14 last 11.23)
 - mkdir 297 (time 1787485844.60 total 30.18 last 7.57)
 - mkdir 461 (time 1787485854.68 total 40.25 last 16.28)
 - mkdir 578 (time 1787485864.71 total 50.29 last 11.66)
 - mkdir 669 (time 1787485874.72 total 60.30 last 9.09)
 - mkdir 799 (time 1787485884.76 total 70.34 last 12.95)
 - mkdir 945 (time 1787485894.80 total 80.38 last 14.53)
total: 1000 mkdir in 85.93 seconds: 11.64 ops/second
 - mkdir 110 (time 1787485823.51 total 10.03 last 10.97)
 - mkdir 216 (time 1787485833.54 total 20.06 last 10.56)
 - mkdir 282 (time 1787485843.55 total 30.07 last 6.59)
 - mkdir 449 (time 1787485853.71 total 40.23 last 16.43)
 - mkdir 570 (time 1787485863.77 total 50.29 last 12.03)
 - mkdir 660 (time 1787485873.83 total 60.35 last 8.95)
 - mkdir 778 (time 1787485883.89 total 70.40 last 11.74)
 - mkdir 928 (time 1787485893.95 total 80.47 last 14.91)
total: 1000 mkdir in 87.42 seconds: 11.44 ops/second
 - mkdir 105 (time 1787485824.27 total 10.07 last 10.42)
 - mkdir 215 (time 1787485834.35 total 20.15 last 10.91)
 - mkdir 293 (time 1787485844.40 total 30.21 last 7.75)
 - mkdir 450 (time 1787485854.45 total 40.25 last 15.64)
 - mkdir 571 (time 1787485864.52 total 50.33 last 12.01)
 - mkdir 657 (time 1787485874.55 total 60.35 last 8.58)
 - mkdir 783 (time 1787485884.58 total 70.39 last 12.56)
 - mkdir 935 (time 1787485894.61 total 80.42 last 15.15)
total: 1000 mkdir in 87.29 seconds: 11.46 ops/second
 - mkdir 107 (time 1787485824.14 total 10.04 last 10.66)
 - mkdir 216 (time 1787485834.23 total 20.13 last 10.80)
 - mkdir 292 (time 1787485844.31 total 30.21 last 7.54)
 - mkdir 450 (time 1787485854.40 total 40.30 last 15.65)
 - mkdir 567 (time 1787485864.46 total 50.36 last 11.64)
 - mkdir 657 (time 1787485874.47 total 60.37 last 8.99)
 - mkdir 780 (time 1787485884.48 total 70.38 last 12.29)
 - mkdir 931 (time 1787485894.49 total 80.39 last 15.08)
total: 1000 mkdir in 87.39 seconds: 11.44 ops/second
 - mkdir 100 (time 1787485824.50 total 10.06 last 9.94)
 - mkdir 199 (time 1787485834.52 total 20.09 last 9.87)
 - mkdir 271 (time 1787485844.57 total 30.13 last 7.17)
 - mkdir 434 (time 1787485854.61 total 40.17 last 16.24)
 - mkdir 556 (time 1787485864.62 total 50.19 last 12.18)
 - mkdir 646 (time 1787485874.67 total 60.23 last 8.96)
 - mkdir 769 (time 1787485884.70 total 70.26 last 12.27)
 - mkdir 912 (time 1787485894.77 total 80.33 last 14.19)
total: 1000 mkdir in 89.76 seconds: 11.14 ops/second
Time for file creation 103 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.66 seconds: 15.06 ops/second
 - mkdir 190 (time 1787486380.48 total 10.05 last 18.90)
 - mkdir 376 (time 1787486390.51 total 20.09 last 18.53)
 - mkdir 541 (time 1787486400.56 total 30.13 last 16.43)
 - mkdir 728 (time 1787486410.57 total 40.15 last 18.68)
 - mkdir 922 (time 1787486420.58 total 50.16 last 19.38)
total: 1000 mkdir in 55.17 seconds: 18.13 ops/second
 - mkdir 184 (time 1787486379.73 total 10.01 last 18.39)
 - mkdir 362 (time 1787486389.76 total 20.03 last 17.76)
 - mkdir 539 (time 1787486399.80 total 30.07 last 17.62)
 - mkdir 722 (time 1787486409.86 total 40.14 last 18.19)
 - mkdir 904 (time 1787486419.90 total 50.17 last 18.14)
total: 1000 mkdir in 56.07 seconds: 17.84 ops/second
 - mkdir 173 (time 1787486377.69 total 10.00 last 17.30)
 - mkdir 330 (time 1787486387.71 total 20.02 last 15.67)
 - mkdir 497 (time 1787486397.76 total 30.07 last 16.62)
 - mkdir 684 (time 1787486407.82 total 40.13 last 18.60)
 - mkdir 857 (time 1787486417.83 total 50.13 last 17.29)
total: 1000 mkdir in 58.24 seconds: 17.17 ops/second
 - mkdir 179 (time 1787486378.81 total 10.02 last 17.86)
 - mkdir 344 (time 1787486388.88 total 20.09 last 16.39)
 - mkdir 519 (time 1787486398.90 total 30.12 last 17.46)
 - mkdir 698 (time 1787486408.90 total 40.12 last 17.90)
 - mkdir 878 (time 1787486418.91 total 50.13 last 17.98)
total: 1000 mkdir in 57.95 seconds: 17.26 ops/second
 - mkdir 186 (time 1787486380.30 total 10.04 last 18.52)
 - mkdir 364 (time 1787486390.33 total 20.07 last 17.75)
 - mkdir 519 (time 1787486400.35 total 30.09 last 15.47)
 - mkdir 701 (time 1787486410.41 total 40.15 last 18.09)
 - mkdir 894 (time 1787486420.42 total 50.16 last 19.28)
total: 1000 mkdir in 57.44 seconds: 17.41 ops/second
 - mkdir 190 (time 1787486380.85 total 10.01 last 18.99)
 - mkdir 370 (time 1787486390.89 total 20.05 last 17.92)
 - mkdir 529 (time 1787486400.95 total 30.11 last 15.80)
 - mkdir 714 (time 1787486410.96 total 40.12 last 18.50)
 - mkdir 915 (time 1787486421.04 total 50.20 last 19.93)
total: 1000 mkdir in 56.89 seconds: 17.58 ops/second
 - mkdir 183 (time 1787486380.09 total 10.05 last 18.21)
 - mkdir 355 (time 1787486390.13 total 20.08 last 17.14)
 - mkdir 520 (time 1787486400.15 total 30.10 last 16.47)
 - mkdir 703 (time 1787486410.15 total 40.10 last 18.30)
 - mkdir 889 (time 1787486420.16 total 50.11 last 18.58)
total: 1000 mkdir in 57.87 seconds: 17.28 ops/second
 - mkdir 185 (time 1787486380.59 total 10.01 last 18.48)
 - mkdir 362 (time 1787486390.60 total 20.02 last 17.69)
 - mkdir 522 (time 1787486400.60 total 30.02 last 16.00)
 - mkdir 705 (time 1787486410.61 total 40.03 last 18.28)
 - mkdir 896 (time 1787486420.63 total 50.05 last 19.06)
total: 1000 mkdir in 58.24 seconds: 17.17 ops/second
 - mkdir 173 (time 1787486379.48 total 10.04 last 17.22)
 - mkdir 340 (time 1787486389.49 total 20.05 last 16.69)
 - mkdir 507 (time 1787486399.57 total 30.13 last 16.57)
 - mkdir 681 (time 1787486409.57 total 40.13 last 17.39)
 - mkdir 863 (time 1787486419.61 total 50.17 last 18.13)
total: 1000 mkdir in 60.14 seconds: 16.63 ops/second
 - mkdir 186 (time 1787486381.22 total 10.16 last 18.31)
 - mkdir 367 (time 1787486391.24 total 20.18 last 18.06)
 - mkdir 527 (time 1787486401.25 total 30.19 last 15.98)
 - mkdir 710 (time 1787486411.28 total 40.22 last 18.25)
 - mkdir 904 (time 1787486421.31 total 50.25 last 19.34)
total: 1000 mkdir in 58.91 seconds: 16.98 ops/second
Time for file creation 70 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787486437 ; total 0 ; last 0)
total: 1000 unlinks in 26 seconds: 38.461540 unlinks/second
 - unlinked 0 (time 1787486439 ; total 0 ; last 0)
total: 1000 unlinks in 26 seconds: 38.461540 unlinks/second
 - unlinked 0 (time 1787486440 ; total 0 ; last 0)
total: 1000 unlinks in 26 seconds: 38.461540 unlinks/second
 - unlinked 0 (time 1787486441 ; total 0 ; last 0)
total: 1000 unlinks in 25 seconds: 40.000000 unlinks/second
 - unlinked 0 (time 1787486440 ; total 0 ; last 0)
total: 1000 unlinks in 27 seconds: 37.037037 unlinks/second
 - unlinked 0 (time 1787486441 ; total 0 ; last 0)
total: 1000 unlinks in 26 seconds: 38.461540 unlinks/second
 - unlinked 0 (time 1787486441 ; total 0 ; last 0)
total: 1000 unlinks in 26 seconds: 38.461540 unlinks/second
 - unlinked 0 (time 1787486442 ; total 1 ; last 1)
total: 1000 unlinks in 26 seconds: 38.461540 unlinks/second
 - unlinked 0 (time 1787486441 ; total 0 ; last 0)
total: 1000 unlinks in 26 seconds: 38.461540 unlinks/second
 - unlinked 0 (time 1787486442 ; total 0 ; last 0)
total: 1000 unlinks in 26 seconds: 38.461540 unlinks/second
Time for file removal 39 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8