== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 16:04:29 (1786392269)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.57 seconds: 17.46 ops/second
 - mkdir 121 (time 1786392290.10 total 10.01 last 12.09)
 - mkdir 280 (time 1786392300.11 total 20.02 last 15.88)
 - mkdir 446 (time 1786392310.13 total 30.04 last 16.57)
 - mkdir 615 (time 1786392320.18 total 40.08 last 16.82)
 - mkdir 788 (time 1786392330.21 total 50.12 last 17.24)
 - mkdir 965 (time 1786392340.25 total 60.16 last 17.63)
total: 1000 mkdir in 62.48 seconds: 16.01 ops/second
 - mkdir 136 (time 1786392293.72 total 10.02 last 13.57)
 - mkdir 307 (time 1786392303.74 total 20.05 last 17.06)
 - mkdir 463 (time 1786392313.77 total 30.07 last 15.56)
 - mkdir 633 (time 1786392323.78 total 40.08 last 16.98)
 - mkdir 812 (time 1786392333.82 total 50.12 last 17.83)
 - mkdir 966 (time 1786392343.88 total 60.19 last 15.30)
total: 1000 mkdir in 65.35 seconds: 15.30 ops/second
 - mkdir 133 (time 1786392294.57 total 10.03 last 13.26)
 - mkdir 310 (time 1786392304.59 total 20.05 last 17.67)
 - mkdir 470 (time 1786392314.61 total 30.07 last 15.96)
 - mkdir 628 (time 1786392324.62 total 40.08 last 15.78)
 - mkdir 819 (time 1786392334.64 total 50.10 last 19.06)
 - mkdir 956 (time 1786392344.70 total 60.16 last 13.61)
total: 1000 mkdir in 66.93 seconds: 14.94 ops/second
 - mkdir 144 (time 1786392295.17 total 10.00 last 14.40)
 - mkdir 315 (time 1786392305.20 total 20.03 last 17.06)
 - mkdir 477 (time 1786392315.20 total 30.03 last 16.19)
 - mkdir 630 (time 1786392325.22 total 40.05 last 15.28)
 - mkdir 823 (time 1786392335.25 total 50.08 last 19.23)
 - mkdir 959 (time 1786392345.26 total 60.09 last 13.59)
total: 1000 mkdir in 67.43 seconds: 14.83 ops/second
 - mkdir 132 (time 1786392294.48 total 10.04 last 13.15)
 - mkdir 295 (time 1786392304.52 total 20.07 last 16.25)
 - mkdir 453 (time 1786392314.55 total 30.11 last 15.74)
 - mkdir 614 (time 1786392324.56 total 40.12 last 16.09)
 - mkdir 806 (time 1786392334.59 total 50.15 last 19.13)
 - mkdir 948 (time 1786392344.71 total 60.27 last 14.03)
total: 1000 mkdir in 68.34 seconds: 14.63 ops/second
 - mkdir 139 (time 1786392295.52 total 10.08 last 13.78)
 - mkdir 315 (time 1786392305.55 total 20.12 last 17.53)
 - mkdir 489 (time 1786392315.57 total 30.14 last 17.37)
 - mkdir 641 (time 1786392325.61 total 40.17 last 15.15)
 - mkdir 830 (time 1786392335.64 total 50.21 last 18.84)
 - mkdir 956 (time 1786392345.67 total 60.24 last 12.56)
total: 1000 mkdir in 67.67 seconds: 14.78 ops/second
 - mkdir 142 (time 1786392295.41 total 10.00 last 14.20)
 - mkdir 314 (time 1786392305.47 total 20.07 last 17.09)
 - mkdir 480 (time 1786392315.51 total 30.10 last 16.55)
 - mkdir 630 (time 1786392325.54 total 40.13 last 14.96)
 - mkdir 821 (time 1786392335.56 total 50.15 last 19.06)
 - mkdir 954 (time 1786392345.67 total 60.26 last 13.16)
total: 1000 mkdir in 67.98 seconds: 14.71 ops/second
 - mkdir 136 (time 1786392295.16 total 10.03 last 13.56)
 - mkdir 304 (time 1786392305.18 total 20.04 last 16.78)
 - mkdir 469 (time 1786392315.19 total 30.06 last 16.48)
 - mkdir 618 (time 1786392325.21 total 40.08 last 14.86)
 - mkdir 811 (time 1786392335.26 total 50.13 last 19.21)
 - mkdir 949 (time 1786392345.31 total 60.18 last 13.73)
total: 1000 mkdir in 68.63 seconds: 14.57 ops/second
 - mkdir 141 (time 1786392295.37 total 10.09 last 13.98)
 - mkdir 308 (time 1786392305.41 total 20.13 last 16.64)
 - mkdir 476 (time 1786392315.43 total 30.14 last 16.78)
 - mkdir 631 (time 1786392325.49 total 40.20 last 15.40)
 - mkdir 818 (time 1786392335.52 total 50.23 last 18.64)
 - mkdir 949 (time 1786392345.56 total 60.27 last 13.06)
total: 1000 mkdir in 68.84 seconds: 14.53 ops/second
 - mkdir 129 (time 1786392294.73 total 10.02 last 12.88)
 - mkdir 293 (time 1786392304.81 total 20.10 last 16.27)
 - mkdir 448 (time 1786392314.85 total 30.14 last 15.43)
 - mkdir 605 (time 1786392324.93 total 40.23 last 15.57)
 - mkdir 798 (time 1786392335.03 total 50.32 last 19.12)
 - mkdir 932 (time 1786392345.04 total 60.33 last 13.39)
 - mkdir 999 (time 1786392355.10 total 70.39 last 6.66)
total: 1000 mkdir in 70.39 seconds: 14.21 ops/second
Time for file creation 84 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
Time for file removal 14 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.65 seconds: 15.34 ops/second
 - mkdir 128 (time 1786392881.46 total 10.02 last 12.78)
 - mkdir 323 (time 1786392891.49 total 20.06 last 19.43)
 - mkdir 453 (time 1786392901.57 total 30.13 last 12.90)
 - mkdir 574 (time 1786392911.63 total 40.19 last 12.03)
 - mkdir 706 (time 1786392921.67 total 50.23 last 13.15)
 - mkdir 840 (time 1786392931.75 total 60.31 last 13.28)
 - mkdir 989 (time 1786392941.78 total 70.34 last 14.86)
total: 1000 mkdir in 71.02 seconds: 14.08 ops/second
 - mkdir 143 (time 1786392883.28 total 10.09 last 14.18)
 - mkdir 327 (time 1786392893.31 total 20.12 last 18.34)
 - mkdir 452 (time 1786392903.36 total 30.17 last 12.44)
 - mkdir 577 (time 1786392913.39 total 40.20 last 12.46)
 - mkdir 715 (time 1786392923.50 total 50.31 last 13.65)
 - mkdir 853 (time 1786392933.58 total 60.39 last 13.68)
 - mkdir 997 (time 1786392943.68 total 70.49 last 14.26)
total: 1000 mkdir in 70.65 seconds: 14.15 ops/second
 - mkdir 148 (time 1786392883.18 total 10.04 last 14.75)
 - mkdir 328 (time 1786392893.27 total 20.13 last 17.84)
 - mkdir 449 (time 1786392903.27 total 30.13 last 12.09)
 - mkdir 573 (time 1786392913.34 total 40.20 last 12.32)
 - mkdir 710 (time 1786392923.42 total 50.28 last 13.59)
 - mkdir 844 (time 1786392933.43 total 60.29 last 13.39)
 - mkdir 988 (time 1786392943.55 total 70.41 last 14.23)
total: 1000 mkdir in 71.53 seconds: 13.98 ops/second
 - mkdir 149 (time 1786392883.19 total 10.11 last 14.73)
 - mkdir 333 (time 1786392893.19 total 20.12 last 18.39)
 - mkdir 450 (time 1786392903.20 total 30.13 last 11.69)
 - mkdir 571 (time 1786392913.30 total 40.23 last 11.98)
 - mkdir 702 (time 1786392923.35 total 50.28 last 13.03)
 - mkdir 843 (time 1786392933.45 total 60.38 last 13.97)
 - mkdir 985 (time 1786392943.53 total 70.45 last 14.09)
total: 1000 mkdir in 72.13 seconds: 13.86 ops/second
 - mkdir 146 (time 1786392883.54 total 10.04 last 14.54)
 - mkdir 325 (time 1786392893.54 total 20.04 last 17.90)
 - mkdir 448 (time 1786392903.55 total 30.05 last 12.29)
 - mkdir 568 (time 1786392913.61 total 40.11 last 11.93)
 - mkdir 702 (time 1786392923.70 total 50.20 last 13.28)
 - mkdir 841 (time 1786392933.72 total 60.23 last 13.87)
 - mkdir 980 (time 1786392943.77 total 70.27 last 13.84)
total: 1000 mkdir in 73.19 seconds: 13.66 ops/second
 - mkdir 145 (time 1786392883.06 total 10.06 last 14.42)
 - mkdir 326 (time 1786392893.11 total 20.11 last 18.00)
 - mkdir 444 (time 1786392903.12 total 30.12 last 11.79)
 - mkdir 564 (time 1786392913.30 total 40.30 last 11.79)
 - mkdir 693 (time 1786392923.34 total 50.34 last 12.85)
 - mkdir 833 (time 1786392933.37 total 60.37 last 13.96)
 - mkdir 977 (time 1786392943.46 total 70.47 last 14.26)
total: 1000 mkdir in 73.82 seconds: 13.55 ops/second
 - mkdir 143 (time 1786392883.17 total 10.06 last 14.21)
 - mkdir 321 (time 1786392893.21 total 20.10 last 17.73)
 - mkdir 443 (time 1786392903.27 total 30.17 last 12.12)
 - mkdir 566 (time 1786392913.30 total 40.19 last 12.27)
 - mkdir 692 (time 1786392923.33 total 50.22 last 12.56)
 - mkdir 833 (time 1786392933.44 total 60.34 last 13.94)
 - mkdir 974 (time 1786392943.48 total 70.37 last 14.05)
total: 1000 mkdir in 74.12 seconds: 13.49 ops/second
 - mkdir 135 (time 1786392882.64 total 10.03 last 13.46)
 - mkdir 315 (time 1786392892.65 total 20.04 last 17.98)
 - mkdir 437 (time 1786392902.67 total 30.06 last 12.17)
 - mkdir 556 (time 1786392912.70 total 40.10 last 11.86)
 - mkdir 682 (time 1786392922.77 total 50.16 last 12.52)
 - mkdir 815 (time 1786392932.78 total 60.17 last 13.29)
 - mkdir 965 (time 1786392942.81 total 70.20 last 14.95)
total: 1000 mkdir in 74.85 seconds: 13.36 ops/second
 - mkdir 141 (time 1786392883.48 total 10.01 last 14.08)
 - mkdir 315 (time 1786392893.49 total 20.02 last 17.39)
 - mkdir 431 (time 1786392903.51 total 30.04 last 11.58)
 - mkdir 547 (time 1786392913.60 total 40.13 last 11.50)
 - mkdir 668 (time 1786392923.65 total 50.18 last 12.04)
 - mkdir 801 (time 1786392933.70 total 60.23 last 13.24)
 - mkdir 949 (time 1786392943.78 total 70.31 last 14.67)
total: 1000 mkdir in 76.70 seconds: 13.04 ops/second
 - mkdir 117 (time 1786392881.00 total 10.04 last 11.65)
 - mkdir 295 (time 1786392891.02 total 20.06 last 17.77)
 - mkdir 416 (time 1786392901.11 total 30.15 last 11.99)
 - mkdir 526 (time 1786392911.14 total 40.18 last 10.97)
 - mkdir 647 (time 1786392921.16 total 50.20 last 12.07)
 - mkdir 768 (time 1786392931.17 total 60.21 last 12.09)
 - mkdir 917 (time 1786392941.18 total 70.22 last 14.89)
total: 1000 mkdir in 79.29 seconds: 12.61 ops/second
Time for file creation 90 sec for 1 RPC sent at a time
 - unlinked 0 (time 1786392959 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1786392960 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1786392960 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1786392960 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1786392960 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1786392960 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1786392961 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1786392961 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1786392961 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1786392961 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
Time for file removal 48 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8