== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 06:21:43 (1789467703)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.40 seconds: 25.30 ops/second
 - mkdir 153 (time 1789467725.12 total 10.04 last 15.23)
 - mkdir 302 (time 1789467735.17 total 20.09 last 14.83)
 - mkdir 441 (time 1789467745.18 total 30.10 last 13.89)
 - mkdir 591 (time 1789467755.18 total 40.10 last 14.99)
 - mkdir 740 (time 1789467765.19 total 50.11 last 14.89)
 - mkdir 883 (time 1789467775.20 total 60.13 last 14.27)
total: 1000 mkdir in 68.34 seconds: 14.63 ops/second
 - mkdir 140 (time 1789467725.33 total 10.03 last 13.96)
 - mkdir 287 (time 1789467735.42 total 20.12 last 14.57)
 - mkdir 433 (time 1789467745.52 total 30.22 last 14.45)
 - mkdir 588 (time 1789467755.52 total 40.22 last 15.50)
 - mkdir 739 (time 1789467765.56 total 50.26 last 15.04)
 - mkdir 879 (time 1789467775.63 total 60.33 last 13.91)
total: 1000 mkdir in 69.89 seconds: 14.31 ops/second
 - mkdir 143 (time 1789467725.90 total 10.05 last 14.23)
 - mkdir 297 (time 1789467735.94 total 20.08 last 15.34)
 - mkdir 434 (time 1789467745.94 total 30.09 last 13.70)
 - mkdir 591 (time 1789467755.94 total 40.09 last 15.70)
 - mkdir 740 (time 1789467765.98 total 50.12 last 14.84)
 - mkdir 886 (time 1789467776.00 total 60.15 last 14.56)
total: 1000 mkdir in 69.41 seconds: 14.41 ops/second
 - mkdir 138 (time 1789467726.14 total 10.01 last 13.79)
 - mkdir 290 (time 1789467736.18 total 20.05 last 15.13)
 - mkdir 429 (time 1789467746.22 total 30.09 last 13.84)
 - mkdir 583 (time 1789467756.24 total 40.11 last 15.38)
 - mkdir 743 (time 1789467766.25 total 50.12 last 15.98)
 - mkdir 883 (time 1789467776.29 total 60.16 last 13.94)
total: 1000 mkdir in 69.94 seconds: 14.30 ops/second
 - mkdir 146 (time 1789467726.69 total 10.04 last 14.54)
 - mkdir 296 (time 1789467736.69 total 20.05 last 14.99)
 - mkdir 442 (time 1789467746.72 total 30.07 last 14.57)
 - mkdir 593 (time 1789467756.74 total 40.10 last 15.06)
 - mkdir 750 (time 1789467766.75 total 50.10 last 15.70)
 - mkdir 891 (time 1789467776.82 total 60.17 last 13.99)
total: 1000 mkdir in 69.59 seconds: 14.37 ops/second
 - mkdir 146 (time 1789467726.98 total 10.06 last 14.52)
 - mkdir 296 (time 1789467737.05 total 20.13 last 14.89)
 - mkdir 442 (time 1789467747.11 total 30.19 last 14.51)
 - mkdir 591 (time 1789467757.13 total 40.21 last 14.87)
 - mkdir 752 (time 1789467767.22 total 50.30 last 15.96)
 - mkdir 890 (time 1789467777.29 total 60.37 last 13.70)
total: 1000 mkdir in 70.13 seconds: 14.26 ops/second
 - mkdir 149 (time 1789467726.84 total 10.04 last 14.84)
 - mkdir 294 (time 1789467736.87 total 20.07 last 14.45)
 - mkdir 440 (time 1789467746.88 total 30.09 last 14.58)
 - mkdir 584 (time 1789467756.89 total 40.09 last 14.39)
 - mkdir 739 (time 1789467766.91 total 50.11 last 15.47)
 - mkdir 875 (time 1789467776.95 total 60.15 last 13.54)
 - mkdir 993 (time 1789467787.01 total 70.21 last 11.74)
total: 1000 mkdir in 71.11 seconds: 14.06 ops/second
 - mkdir 148 (time 1789467726.96 total 10.10 last 14.66)
 - mkdir 299 (time 1789467737.07 total 20.21 last 14.94)
 - mkdir 443 (time 1789467747.14 total 30.28 last 14.30)
 - mkdir 587 (time 1789467757.15 total 40.29 last 14.38)
 - mkdir 741 (time 1789467767.20 total 50.34 last 15.32)
 - mkdir 882 (time 1789467777.24 total 60.37 last 14.05)
 - mkdir 993 (time 1789467787.26 total 70.40 last 11.07)
total: 1000 mkdir in 71.36 seconds: 14.01 ops/second
 - mkdir 141 (time 1789467727.22 total 10.09 last 13.97)
 - mkdir 288 (time 1789467737.23 total 20.10 last 14.68)
 - mkdir 433 (time 1789467747.24 total 30.12 last 14.48)
 - mkdir 582 (time 1789467757.32 total 40.19 last 14.79)
 - mkdir 740 (time 1789467767.33 total 50.21 last 15.78)
 - mkdir 883 (time 1789467777.41 total 60.28 last 14.19)
 - mkdir 991 (time 1789467787.43 total 70.31 last 10.77)
total: 1000 mkdir in 71.44 seconds: 14.00 ops/second
 - mkdir 143 (time 1789467726.47 total 10.02 last 14.28)
 - mkdir 285 (time 1789467736.49 total 20.04 last 14.17)
 - mkdir 419 (time 1789467746.50 total 30.05 last 13.37)
 - mkdir 569 (time 1789467756.53 total 40.08 last 14.96)
 - mkdir 727 (time 1789467766.57 total 50.12 last 15.74)
 - mkdir 870 (time 1789467776.65 total 60.20 last 14.19)
 - mkdir 980 (time 1789467786.81 total 70.36 last 10.82)
total: 1000 mkdir in 72.93 seconds: 13.71 ops/second
Time for file creation 84 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 17 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.65 seconds: 15.34 ops/second
 - mkdir 148 (time 1789468302.49 total 10.03 last 14.75)
 - mkdir 317 (time 1789468312.51 total 20.05 last 16.87)
 - mkdir 495 (time 1789468322.67 total 30.21 last 17.52)
 - mkdir 662 (time 1789468332.71 total 40.25 last 16.63)
 - mkdir 796 (time 1789468342.79 total 50.33 last 13.30)
 - mkdir 962 (time 1789468352.80 total 60.34 last 16.59)
total: 1000 mkdir in 63.08 seconds: 15.85 ops/second
 - mkdir 146 (time 1789468302.74 total 10.08 last 14.48)
 - mkdir 311 (time 1789468312.75 total 20.09 last 16.48)
 - mkdir 492 (time 1789468322.77 total 30.12 last 18.06)
 - mkdir 655 (time 1789468332.80 total 40.14 last 16.26)
 - mkdir 787 (time 1789468342.89 total 50.24 last 13.08)
 - mkdir 954 (time 1789468352.94 total 60.28 last 16.63)
total: 1000 mkdir in 64.69 seconds: 15.46 ops/second
 - mkdir 162 (time 1789468304.25 total 10.03 last 16.16)
 - mkdir 329 (time 1789468314.30 total 20.07 last 16.62)
 - mkdir 500 (time 1789468324.37 total 30.14 last 16.99)
 - mkdir 675 (time 1789468334.38 total 40.15 last 17.48)
 - mkdir 812 (time 1789468344.44 total 50.22 last 13.61)
 - mkdir 969 (time 1789468354.46 total 60.23 last 15.68)
total: 1000 mkdir in 64.19 seconds: 15.58 ops/second
 - mkdir 176 (time 1789468305.62 total 10.02 last 17.57)
 - mkdir 350 (time 1789468315.63 total 20.03 last 17.38)
 - mkdir 520 (time 1789468325.66 total 30.06 last 16.95)
 - mkdir 691 (time 1789468335.74 total 40.14 last 16.95)
 - mkdir 828 (time 1789468345.87 total 50.27 last 13.52)
 - mkdir 982 (time 1789468356.00 total 60.40 last 15.21)
total: 1000 mkdir in 63.01 seconds: 15.87 ops/second
 - mkdir 178 (time 1789468305.66 total 10.00 last 17.80)
 - mkdir 349 (time 1789468315.73 total 20.08 last 16.97)
 - mkdir 511 (time 1789468325.74 total 30.08 last 16.19)
 - mkdir 683 (time 1789468335.79 total 40.13 last 17.12)
 - mkdir 822 (time 1789468345.85 total 50.19 last 13.82)
 - mkdir 981 (time 1789468355.86 total 60.21 last 15.87)
total: 1000 mkdir in 63.49 seconds: 15.75 ops/second
 - mkdir 172 (time 1789468305.40 total 10.00 last 17.20)
 - mkdir 335 (time 1789468315.41 total 20.01 last 16.29)
 - mkdir 501 (time 1789468325.42 total 30.02 last 16.58)
 - mkdir 675 (time 1789468335.48 total 40.08 last 17.30)
 - mkdir 814 (time 1789468345.55 total 50.15 last 13.80)
 - mkdir 973 (time 1789468355.58 total 60.18 last 15.85)
total: 1000 mkdir in 64.49 seconds: 15.51 ops/second
 - mkdir 175 (time 1789468305.40 total 10.08 last 17.35)
 - mkdir 334 (time 1789468315.41 total 20.09 last 15.90)
 - mkdir 499 (time 1789468325.44 total 30.12 last 16.45)
 - mkdir 675 (time 1789468335.51 total 40.19 last 17.47)
 - mkdir 811 (time 1789468345.57 total 50.25 last 13.53)
 - mkdir 969 (time 1789468355.61 total 60.29 last 15.73)
total: 1000 mkdir in 65.36 seconds: 15.30 ops/second
 - mkdir 152 (time 1789468304.33 total 10.04 last 15.15)
 - mkdir 307 (time 1789468314.34 total 20.05 last 15.48)
 - mkdir 484 (time 1789468324.39 total 30.10 last 17.60)
 - mkdir 657 (time 1789468334.43 total 40.13 last 17.24)
 - mkdir 792 (time 1789468344.43 total 50.14 last 13.50)
 - mkdir 952 (time 1789468354.46 total 60.17 last 15.95)
total: 1000 mkdir in 67.18 seconds: 14.89 ops/second
 - mkdir 164 (time 1789468305.29 total 10.02 last 16.37)
 - mkdir 322 (time 1789468315.37 total 20.10 last 15.67)
 - mkdir 484 (time 1789468325.44 total 30.17 last 16.09)
 - mkdir 664 (time 1789468335.48 total 40.21 last 17.92)
 - mkdir 800 (time 1789468345.56 total 50.29 last 13.49)
 - mkdir 960 (time 1789468355.57 total 60.30 last 15.99)
total: 1000 mkdir in 66.61 seconds: 15.01 ops/second
 - mkdir 168 (time 1789468305.68 total 10.08 last 16.67)
 - mkdir 320 (time 1789468315.73 total 20.12 last 15.13)
 - mkdir 472 (time 1789468325.76 total 30.16 last 15.15)
 - mkdir 641 (time 1789468335.84 total 40.24 last 16.77)
 - mkdir 775 (time 1789468345.89 total 50.28 last 13.34)
 - mkdir 931 (time 1789468355.91 total 60.30 last 15.57)
total: 1000 mkdir in 68.20 seconds: 14.66 ops/second
Time for file creation 81 sec for 1 RPC sent at a time
 - unlinked 0 (time 1789468371 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1789468373 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1789468373 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1789468373 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789468374 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789468374 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789468374 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789468374 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789468374 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789468374 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
Time for file removal 44 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8