== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 00:27:24 (1787891244)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.63 seconds: 15.88 ops/second
 - mkdir 175 (time 1787891270.85 total 10.03 last 17.44)
 - mkdir 327 (time 1787891280.89 total 20.07 last 15.14)
 - mkdir 456 (time 1787891290.91 total 30.09 last 12.87)
 - mkdir 604 (time 1787891300.96 total 40.15 last 14.72)
 - mkdir 724 (time 1787891310.97 total 50.15 last 11.99)
 - mkdir 887 (time 1787891321.05 total 60.24 last 16.17)
total: 1000 mkdir in 68.09 seconds: 14.69 ops/second
 - mkdir 158 (time 1787891270.01 total 10.03 last 15.76)
 - mkdir 313 (time 1787891280.06 total 20.08 last 15.42)
 - mkdir 435 (time 1787891290.08 total 30.09 last 12.18)
 - mkdir 580 (time 1787891300.14 total 40.15 last 14.41)
 - mkdir 695 (time 1787891310.15 total 50.16 last 11.49)
 - mkdir 846 (time 1787891320.18 total 60.20 last 15.05)
 - mkdir 979 (time 1787891330.19 total 70.20 last 13.29)
total: 1000 mkdir in 72.14 seconds: 13.86 ops/second
 - mkdir 177 (time 1787891271.41 total 10.01 last 17.68)
 - mkdir 326 (time 1787891281.48 total 20.08 last 14.80)
 - mkdir 449 (time 1787891291.50 total 30.10 last 12.28)
 - mkdir 589 (time 1787891301.60 total 40.20 last 13.86)
 - mkdir 705 (time 1787891311.65 total 50.25 last 11.54)
 - mkdir 865 (time 1787891321.67 total 60.27 last 15.98)
 - mkdir 996 (time 1787891331.76 total 70.37 last 12.97)
total: 1000 mkdir in 70.81 seconds: 14.12 ops/second
 - mkdir 180 (time 1787891272.20 total 10.03 last 17.94)
 - mkdir 325 (time 1787891282.22 total 20.05 last 14.48)
 - mkdir 451 (time 1787891292.39 total 30.22 last 12.39)
 - mkdir 593 (time 1787891302.47 total 40.31 last 14.08)
 - mkdir 720 (time 1787891312.48 total 50.31 last 12.69)
 - mkdir 881 (time 1787891322.54 total 60.37 last 16.00)
total: 1000 mkdir in 70.04 seconds: 14.28 ops/second
 - mkdir 175 (time 1787891272.09 total 10.05 last 17.42)
 - mkdir 320 (time 1787891282.10 total 20.06 last 14.48)
 - mkdir 451 (time 1787891292.11 total 30.07 last 13.09)
 - mkdir 589 (time 1787891302.21 total 40.16 last 13.67)
 - mkdir 710 (time 1787891312.23 total 50.19 last 12.07)
 - mkdir 875 (time 1787891322.33 total 60.28 last 16.34)
 - mkdir 996 (time 1787891332.35 total 70.30 last 12.08)
total: 1000 mkdir in 70.88 seconds: 14.11 ops/second
 - mkdir 176 (time 1787891272.33 total 10.01 last 17.58)
 - mkdir 322 (time 1787891282.33 total 20.02 last 14.59)
 - mkdir 450 (time 1787891292.39 total 30.07 last 12.73)
 - mkdir 589 (time 1787891302.44 total 40.13 last 13.82)
 - mkdir 713 (time 1787891312.48 total 50.16 last 12.36)
 - mkdir 872 (time 1787891322.55 total 60.24 last 15.78)
 - mkdir 996 (time 1787891332.72 total 70.40 last 12.20)
total: 1000 mkdir in 70.90 seconds: 14.10 ops/second
 - mkdir 174 (time 1787891272.24 total 10.04 last 17.33)
 - mkdir 320 (time 1787891282.28 total 20.08 last 14.54)
 - mkdir 448 (time 1787891292.47 total 30.27 last 12.56)
 - mkdir 585 (time 1787891302.47 total 40.27 last 13.70)
 - mkdir 709 (time 1787891312.55 total 50.36 last 12.30)
 - mkdir 870 (time 1787891322.55 total 60.36 last 16.10)
 - mkdir 990 (time 1787891332.61 total 70.41 last 11.93)
total: 1000 mkdir in 71.60 seconds: 13.97 ops/second
 - mkdir 162 (time 1787891271.93 total 10.00 last 16.19)
 - mkdir 307 (time 1787891282.07 total 20.14 last 14.31)
 - mkdir 438 (time 1787891292.12 total 30.19 last 13.03)
 - mkdir 574 (time 1787891302.21 total 40.28 last 13.48)
 - mkdir 695 (time 1787891312.28 total 50.35 last 12.01)
 - mkdir 851 (time 1787891322.33 total 60.40 last 15.52)
 - mkdir 974 (time 1787891332.35 total 70.42 last 12.28)
total: 1000 mkdir in 74.13 seconds: 13.49 ops/second
 - mkdir 164 (time 1787891272.06 total 10.01 last 16.38)
 - mkdir 307 (time 1787891282.11 total 20.06 last 14.22)
 - mkdir 431 (time 1787891292.18 total 30.13 last 12.32)
 - mkdir 566 (time 1787891302.24 total 40.19 last 13.42)
 - mkdir 685 (time 1787891312.28 total 50.23 last 11.85)
 - mkdir 838 (time 1787891322.32 total 60.27 last 15.25)
 - mkdir 960 (time 1787891332.46 total 70.41 last 12.03)
total: 1000 mkdir in 75.28 seconds: 13.28 ops/second
 - mkdir 170 (time 1787891272.79 total 10.08 last 16.86)
 - mkdir 303 (time 1787891282.80 total 20.09 last 13.29)
 - mkdir 416 (time 1787891292.81 total 30.10 last 11.29)
 - mkdir 546 (time 1787891302.92 total 40.21 last 12.85)
 - mkdir 670 (time 1787891312.95 total 50.24 last 12.36)
 - mkdir 817 (time 1787891323.00 total 60.29 last 14.62)
 - mkdir 937 (time 1787891333.12 total 70.41 last 11.87)
total: 1000 mkdir in 76.24 seconds: 13.12 ops/second
Time for file creation 90 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 17 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.70 seconds: 14.24 ops/second
 - mkdir 147 (time 1787891874.74 total 10.02 last 14.66)
 - mkdir 358 (time 1787891884.85 total 20.14 last 20.87)
 - mkdir 501 (time 1787891894.86 total 30.15 last 14.28)
 - mkdir 693 (time 1787891904.89 total 40.17 last 19.16)
 - mkdir 902 (time 1787891914.95 total 50.23 last 20.77)
total: 1000 mkdir in 55.56 seconds: 18.00 ops/second
 - mkdir 174 (time 1787891878.05 total 10.02 last 17.36)
 - mkdir 361 (time 1787891888.09 total 20.06 last 18.63)
 - mkdir 525 (time 1787891898.12 total 30.09 last 16.35)
 - mkdir 709 (time 1787891908.14 total 40.11 last 18.36)
 - mkdir 932 (time 1787891918.16 total 50.13 last 22.26)
total: 1000 mkdir in 56.06 seconds: 17.84 ops/second
 - mkdir 165 (time 1787891877.76 total 10.01 last 16.49)
 - mkdir 351 (time 1787891887.87 total 20.12 last 18.39)
 - mkdir 517 (time 1787891897.92 total 30.17 last 16.51)
 - mkdir 699 (time 1787891907.92 total 40.18 last 18.20)
 - mkdir 925 (time 1787891917.96 total 50.21 last 22.52)
total: 1000 mkdir in 56.89 seconds: 17.58 ops/second
 - mkdir 163 (time 1787891877.62 total 10.03 last 16.26)
 - mkdir 348 (time 1787891887.66 total 20.06 last 18.43)
 - mkdir 509 (time 1787891897.66 total 30.06 last 16.10)
 - mkdir 692 (time 1787891907.68 total 40.09 last 18.26)
 - mkdir 920 (time 1787891917.76 total 50.17 last 22.62)
total: 1000 mkdir in 57.26 seconds: 17.46 ops/second
 - mkdir 172 (time 1787891878.54 total 10.01 last 17.19)
 - mkdir 355 (time 1787891888.54 total 20.01 last 18.30)
 - mkdir 529 (time 1787891898.58 total 30.04 last 17.34)
 - mkdir 719 (time 1787891908.60 total 40.07 last 18.96)
 - mkdir 933 (time 1787891918.61 total 50.08 last 21.37)
total: 1000 mkdir in 56.65 seconds: 17.65 ops/second
 - mkdir 163 (time 1787891877.94 total 10.02 last 16.27)
 - mkdir 348 (time 1787891887.99 total 20.07 last 18.41)
 - mkdir 515 (time 1787891898.01 total 30.09 last 16.67)
 - mkdir 700 (time 1787891908.02 total 40.09 last 18.49)
 - mkdir 922 (time 1787891918.07 total 50.14 last 22.10)
total: 1000 mkdir in 57.26 seconds: 17.46 ops/second
 - mkdir 164 (time 1787891877.70 total 10.03 last 16.35)
 - mkdir 351 (time 1787891887.74 total 20.07 last 18.63)
 - mkdir 510 (time 1787891897.75 total 30.08 last 15.88)
 - mkdir 695 (time 1787891907.82 total 40.15 last 18.37)
 - mkdir 919 (time 1787891917.87 total 50.20 last 22.30)
total: 1000 mkdir in 57.72 seconds: 17.32 ops/second
 - mkdir 154 (time 1787891877.64 total 10.01 last 15.38)
 - mkdir 338 (time 1787891887.65 total 20.03 last 18.37)
 - mkdir 500 (time 1787891897.68 total 30.06 last 16.15)
 - mkdir 688 (time 1787891907.71 total 40.09 last 18.74)
 - mkdir 909 (time 1787891917.75 total 50.13 last 22.01)
total: 1000 mkdir in 58.74 seconds: 17.03 ops/second
 - mkdir 163 (time 1787891878.28 total 10.05 last 16.22)
 - mkdir 343 (time 1787891888.31 total 20.07 last 17.96)
 - mkdir 509 (time 1787891898.36 total 30.13 last 16.51)
 - mkdir 691 (time 1787891908.40 total 40.17 last 18.13)
 - mkdir 908 (time 1787891918.43 total 50.20 last 21.63)
total: 1000 mkdir in 59.62 seconds: 16.77 ops/second
 - mkdir 159 (time 1787891878.33 total 10.04 last 15.83)
 - mkdir 334 (time 1787891888.40 total 20.11 last 17.38)
 - mkdir 491 (time 1787891898.41 total 30.12 last 15.69)
 - mkdir 678 (time 1787891908.46 total 40.17 last 18.61)
 - mkdir 900 (time 1787891918.52 total 50.24 last 22.05)
 - mkdir 999 (time 1787891928.57 total 60.29 last 9.85)
total: 1000 mkdir in 60.29 seconds: 16.59 ops/second
Time for file creation 73 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787891937 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787891939 ; total 1 ; last 1)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787891939 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787891939 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787891939 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787891940 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787891939 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787891940 ; total 1 ; last 1)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787891939 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787891940 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
Time for file removal 47 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8