== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 14:54:29 (1787338469)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.50 seconds: 19.94 ops/second
 - mkdir 128 (time 1787338490.77 total 10.03 last 12.76)
 - mkdir 255 (time 1787338500.85 total 20.12 last 12.60)
 - mkdir 399 (time 1787338510.87 total 30.14 last 14.37)
 - mkdir 534 (time 1787338520.89 total 40.16 last 13.48)
 - mkdir 663 (time 1787338530.98 total 50.24 last 12.79)
 - mkdir 802 (time 1787338540.99 total 60.26 last 13.88)
 - mkdir 934 (time 1787338551.01 total 70.28 last 13.17)
total: 1000 mkdir in 75.40 seconds: 13.26 ops/second
 - mkdir 129 (time 1787338492.20 total 10.06 last 12.82)
 - mkdir 264 (time 1787338502.21 total 20.08 last 13.48)
 - mkdir 420 (time 1787338512.23 total 30.10 last 15.57)
 - mkdir 544 (time 1787338522.23 total 40.10 last 12.40)
 - mkdir 672 (time 1787338532.38 total 50.25 last 12.61)
 - mkdir 811 (time 1787338542.39 total 60.26 last 13.89)
 - mkdir 951 (time 1787338552.49 total 70.36 last 13.85)
total: 1000 mkdir in 74.08 seconds: 13.50 ops/second
 - mkdir 126 (time 1787338492.43 total 10.03 last 12.56)
 - mkdir 264 (time 1787338502.52 total 20.12 last 13.68)
 - mkdir 417 (time 1787338512.53 total 30.12 last 15.29)
 - mkdir 536 (time 1787338522.68 total 40.27 last 11.72)
 - mkdir 663 (time 1787338532.70 total 50.30 last 12.67)
 - mkdir 809 (time 1787338542.77 total 60.37 last 14.49)
 - mkdir 950 (time 1787338552.79 total 70.39 last 14.07)
total: 1000 mkdir in 74.26 seconds: 13.47 ops/second
 - mkdir 133 (time 1787338492.22 total 10.08 last 13.19)
 - mkdir 265 (time 1787338502.23 total 20.09 last 13.18)
 - mkdir 418 (time 1787338512.24 total 30.10 last 15.29)
 - mkdir 539 (time 1787338522.24 total 40.11 last 12.09)
 - mkdir 671 (time 1787338532.32 total 50.19 last 13.09)
 - mkdir 811 (time 1787338542.33 total 60.19 last 13.99)
 - mkdir 946 (time 1787338552.39 total 70.25 last 13.42)
total: 1000 mkdir in 74.55 seconds: 13.41 ops/second
 - mkdir 127 (time 1787338491.98 total 10.10 last 12.57)
 - mkdir 262 (time 1787338502.01 total 20.13 last 13.46)
 - mkdir 415 (time 1787338512.06 total 30.18 last 15.22)
 - mkdir 535 (time 1787338522.14 total 40.26 last 11.91)
 - mkdir 668 (time 1787338532.16 total 50.28 last 13.27)
 - mkdir 805 (time 1787338542.22 total 60.34 last 13.63)
 - mkdir 940 (time 1787338552.26 total 70.38 last 13.44)
total: 1000 mkdir in 75.11 seconds: 13.31 ops/second
 - mkdir 132 (time 1787338491.85 total 10.08 last 13.09)
 - mkdir 261 (time 1787338501.92 total 20.15 last 12.81)
 - mkdir 413 (time 1787338511.95 total 30.18 last 15.15)
 - mkdir 538 (time 1787338522.03 total 40.26 last 12.40)
 - mkdir 662 (time 1787338532.05 total 50.27 last 12.38)
 - mkdir 800 (time 1787338542.06 total 60.28 last 13.79)
 - mkdir 940 (time 1787338552.08 total 70.31 last 13.96)
total: 1000 mkdir in 75.29 seconds: 13.28 ops/second
 - mkdir 127 (time 1787338491.90 total 10.10 last 12.57)
 - mkdir 256 (time 1787338501.92 total 20.12 last 12.87)
 - mkdir 405 (time 1787338511.92 total 30.13 last 14.90)
 - mkdir 521 (time 1787338521.95 total 40.16 last 11.57)
 - mkdir 650 (time 1787338531.95 total 50.16 last 12.89)
 - mkdir 792 (time 1787338542.06 total 60.26 last 14.06)
 - mkdir 927 (time 1787338552.09 total 70.29 last 13.46)
total: 1000 mkdir in 76.68 seconds: 13.04 ops/second
 - mkdir 121 (time 1787338490.30 total 10.11 last 11.97)
 - mkdir 242 (time 1787338500.37 total 20.17 last 12.02)
 - mkdir 383 (time 1787338510.38 total 30.18 last 14.09)
 - mkdir 513 (time 1787338520.43 total 40.24 last 12.93)
 - mkdir 644 (time 1787338530.47 total 50.27 last 13.05)
 - mkdir 779 (time 1787338540.50 total 60.30 last 13.46)
 - mkdir 914 (time 1787338550.52 total 70.32 last 13.48)
total: 1000 mkdir in 78.33 seconds: 12.77 ops/second
 - mkdir 127 (time 1787338492.24 total 10.04 last 12.65)
 - mkdir 261 (time 1787338502.27 total 20.07 last 13.37)
 - mkdir 410 (time 1787338512.30 total 30.09 last 14.86)
 - mkdir 530 (time 1787338522.32 total 40.11 last 11.98)
 - mkdir 658 (time 1787338532.33 total 50.12 last 12.79)
 - mkdir 794 (time 1787338542.36 total 60.15 last 13.56)
 - mkdir 932 (time 1787338552.38 total 70.18 last 13.76)
total: 1000 mkdir in 76.35 seconds: 13.10 ops/second
 - mkdir 123 (time 1787338491.47 total 10.05 last 12.24)
 - mkdir 252 (time 1787338501.59 total 20.17 last 12.75)
 - mkdir 398 (time 1787338511.61 total 30.19 last 14.57)
 - mkdir 520 (time 1787338521.66 total 40.24 last 12.13)
 - mkdir 645 (time 1787338531.67 total 50.25 last 12.49)
 - mkdir 785 (time 1787338541.74 total 60.32 last 13.90)
 - mkdir 920 (time 1787338551.81 total 70.39 last 13.40)
total: 1000 mkdir in 77.84 seconds: 12.85 ops/second
Time for file creation 90 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 19 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.56 seconds: 17.92 ops/second
 - mkdir 151 (time 1787339176.33 total 10.06 last 15.01)
 - mkdir 315 (time 1787339186.38 total 20.11 last 16.32)
 - mkdir 484 (time 1787339196.41 total 30.14 last 16.86)
 - mkdir 656 (time 1787339206.42 total 40.15 last 17.18)
 - mkdir 813 (time 1787339216.48 total 50.21 last 15.60)
 - mkdir 952 (time 1787339226.50 total 60.23 last 13.87)
total: 1000 mkdir in 63.89 seconds: 15.65 ops/second
 - mkdir 146 (time 1787339175.80 total 10.05 last 14.53)
 - mkdir 311 (time 1787339185.81 total 20.06 last 16.48)
 - mkdir 477 (time 1787339195.87 total 30.12 last 16.50)
 - mkdir 649 (time 1787339205.95 total 40.20 last 17.07)
 - mkdir 808 (time 1787339215.98 total 50.23 last 15.85)
 - mkdir 940 (time 1787339226.04 total 60.29 last 13.13)
total: 1000 mkdir in 66.67 seconds: 15.00 ops/second
 - mkdir 143 (time 1787339176.14 total 10.04 last 14.25)
 - mkdir 299 (time 1787339186.16 total 20.06 last 15.57)
 - mkdir 466 (time 1787339196.21 total 30.11 last 16.61)
 - mkdir 634 (time 1787339206.22 total 40.13 last 16.78)
 - mkdir 792 (time 1787339216.26 total 50.16 last 15.75)
 - mkdir 925 (time 1787339226.32 total 60.22 last 13.22)
total: 1000 mkdir in 69.37 seconds: 14.42 ops/second
 - mkdir 136 (time 1787339175.89 total 10.02 last 13.57)
 - mkdir 291 (time 1787339185.89 total 20.03 last 15.50)
 - mkdir 460 (time 1787339195.91 total 30.05 last 16.86)
 - mkdir 625 (time 1787339205.94 total 40.07 last 16.46)
 - mkdir 783 (time 1787339215.98 total 50.12 last 15.72)
 - mkdir 917 (time 1787339226.05 total 60.19 last 13.31)
 - mkdir 999 (time 1787339236.08 total 70.21 last 8.18)
total: 1000 mkdir in 70.21 seconds: 14.24 ops/second
 - mkdir 140 (time 1787339177.09 total 10.05 last 13.93)
 - mkdir 308 (time 1787339187.10 total 20.07 last 16.77)
 - mkdir 478 (time 1787339197.14 total 30.11 last 16.94)
 - mkdir 645 (time 1787339207.20 total 40.17 last 16.60)
 - mkdir 799 (time 1787339217.23 total 50.20 last 15.36)
 - mkdir 925 (time 1787339227.26 total 60.23 last 12.56)
total: 1000 mkdir in 69.48 seconds: 14.39 ops/second
 - mkdir 144 (time 1787339176.85 total 10.13 last 14.22)
 - mkdir 305 (time 1787339186.86 total 20.14 last 16.09)
 - mkdir 471 (time 1787339196.89 total 30.16 last 16.56)
 - mkdir 635 (time 1787339206.95 total 40.22 last 16.30)
 - mkdir 786 (time 1787339216.95 total 50.22 last 15.10)
 - mkdir 918 (time 1787339227.00 total 60.27 last 13.14)
 - mkdir 999 (time 1787339237.34 total 70.62 last 7.83)
total: 1000 mkdir in 70.62 seconds: 14.16 ops/second
 - mkdir 135 (time 1787339175.17 total 10.05 last 13.43)
 - mkdir 291 (time 1787339185.20 total 20.08 last 15.55)
 - mkdir 453 (time 1787339195.31 total 30.19 last 16.02)
 - mkdir 619 (time 1787339205.38 total 40.26 last 16.49)
 - mkdir 770 (time 1787339215.39 total 50.28 last 15.08)
 - mkdir 903 (time 1787339225.49 total 60.37 last 13.18)
 - mkdir 989 (time 1787339235.75 total 70.63 last 8.38)
total: 1000 mkdir in 72.36 seconds: 13.82 ops/second
 - mkdir 140 (time 1787339176.31 total 10.05 last 13.93)
 - mkdir 295 (time 1787339186.32 total 20.07 last 15.47)
 - mkdir 463 (time 1787339196.41 total 30.16 last 16.66)
 - mkdir 630 (time 1787339206.42 total 40.17 last 16.67)
 - mkdir 783 (time 1787339216.46 total 50.21 last 15.25)
 - mkdir 908 (time 1787339226.48 total 60.22 last 12.48)
 - mkdir 990 (time 1787339236.48 total 70.23 last 8.19)
total: 1000 mkdir in 71.77 seconds: 13.93 ops/second
 - mkdir 137 (time 1787339176.40 total 10.02 last 13.68)
 - mkdir 297 (time 1787339186.42 total 20.04 last 15.96)
 - mkdir 462 (time 1787339196.44 total 30.06 last 16.47)
 - mkdir 627 (time 1787339206.49 total 40.11 last 16.42)
 - mkdir 780 (time 1787339216.53 total 50.15 last 15.24)
 - mkdir 910 (time 1787339226.63 total 60.25 last 12.88)
 - mkdir 984 (time 1787339236.81 total 70.43 last 7.27)
total: 1000 mkdir in 72.63 seconds: 13.77 ops/second
 - mkdir 132 (time 1787339176.86 total 10.05 last 13.13)
 - mkdir 288 (time 1787339186.92 total 20.12 last 15.49)
 - mkdir 448 (time 1787339196.93 total 30.13 last 15.98)
 - mkdir 609 (time 1787339206.96 total 40.16 last 16.05)
 - mkdir 750 (time 1787339217.02 total 50.21 last 14.03)
 - mkdir 873 (time 1787339227.09 total 60.29 last 12.20)
 - mkdir 945 (time 1787339237.15 total 70.35 last 7.16)
total: 1000 mkdir in 75.55 seconds: 13.24 ops/second
Time for file creation 87 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787339252 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787339252 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787339253 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787339253 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787339253 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787339253 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787339254 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787339254 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787339254 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787339253 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
Time for file removal 46 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8