== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 03:20:03 (1787728803)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.40 seconds: 25.03 ops/second
 - mkdir 137 (time 1787728826.69 total 10.05 last 13.63)
 - mkdir 286 (time 1787728836.79 total 20.15 last 14.76)
 - mkdir 431 (time 1787728846.86 total 30.21 last 14.40)
 - mkdir 577 (time 1787728856.86 total 40.22 last 14.59)
 - mkdir 718 (time 1787728866.86 total 50.22 last 14.10)
 - mkdir 868 (time 1787728876.90 total 60.26 last 14.94)
total: 1000 mkdir in 69.99 seconds: 14.29 ops/second
 - mkdir 132 (time 1787728826.98 total 10.01 last 13.18)
 - mkdir 280 (time 1787728837.03 total 20.07 last 14.72)
 - mkdir 421 (time 1787728847.06 total 30.10 last 14.06)
 - mkdir 562 (time 1787728857.07 total 40.10 last 14.10)
 - mkdir 708 (time 1787728867.15 total 50.19 last 14.48)
 - mkdir 862 (time 1787728877.16 total 60.20 last 15.39)
 - mkdir 992 (time 1787728887.29 total 70.33 last 12.83)
total: 1000 mkdir in 71.24 seconds: 14.04 ops/second
 - mkdir 131 (time 1787728826.23 total 10.04 last 13.05)
 - mkdir 280 (time 1787728836.31 total 20.12 last 14.77)
 - mkdir 420 (time 1787728846.38 total 30.19 last 13.91)
 - mkdir 565 (time 1787728856.41 total 40.22 last 14.46)
 - mkdir 701 (time 1787728866.41 total 50.22 last 13.60)
 - mkdir 849 (time 1787728876.44 total 60.25 last 14.75)
 - mkdir 985 (time 1787728886.46 total 70.28 last 13.57)
total: 1000 mkdir in 72.10 seconds: 13.87 ops/second
 - mkdir 134 (time 1787728826.77 total 10.04 last 13.34)
 - mkdir 280 (time 1787728836.80 total 20.08 last 14.55)
 - mkdir 423 (time 1787728846.86 total 30.14 last 14.21)
 - mkdir 566 (time 1787728856.88 total 40.16 last 14.28)
 - mkdir 703 (time 1787728866.92 total 50.20 last 13.64)
 - mkdir 851 (time 1787728876.97 total 60.25 last 14.72)
 - mkdir 981 (time 1787728887.01 total 70.29 last 12.95)
total: 1000 mkdir in 72.89 seconds: 13.72 ops/second
 - mkdir 132 (time 1787728826.54 total 10.04 last 13.15)
 - mkdir 275 (time 1787728836.57 total 20.07 last 14.26)
 - mkdir 418 (time 1787728846.64 total 30.13 last 14.21)
 - mkdir 561 (time 1787728856.75 total 40.24 last 14.13)
 - mkdir 695 (time 1787728866.77 total 50.26 last 13.38)
 - mkdir 842 (time 1787728876.81 total 60.30 last 14.64)
 - mkdir 970 (time 1787728886.87 total 70.37 last 12.72)
total: 1000 mkdir in 75.40 seconds: 13.26 ops/second
 - mkdir 135 (time 1787728827.51 total 10.01 last 13.49)
 - mkdir 272 (time 1787728837.56 total 20.05 last 13.64)
 - mkdir 425 (time 1787728847.56 total 30.05 last 15.30)
 - mkdir 565 (time 1787728857.61 total 40.10 last 13.93)
 - mkdir 699 (time 1787728867.68 total 50.17 last 13.31)
 - mkdir 847 (time 1787728877.69 total 60.19 last 14.78)
 - mkdir 976 (time 1787728887.87 total 70.36 last 12.68)
total: 1000 mkdir in 74.60 seconds: 13.40 ops/second
 - mkdir 136 (time 1787728826.01 total 10.01 last 13.59)
 - mkdir 276 (time 1787728836.02 total 20.02 last 13.99)
 - mkdir 413 (time 1787728846.07 total 30.07 last 13.63)
 - mkdir 549 (time 1787728856.08 total 40.07 last 13.59)
 - mkdir 686 (time 1787728866.10 total 50.10 last 13.67)
 - mkdir 830 (time 1787728876.14 total 60.13 last 14.35)
 - mkdir 958 (time 1787728886.15 total 70.15 last 12.78)
total: 1000 mkdir in 76.43 seconds: 13.08 ops/second
 - mkdir 135 (time 1787728827.87 total 10.00 last 13.49)
 - mkdir 274 (time 1787728837.98 total 20.11 last 13.76)
 - mkdir 429 (time 1787728848.02 total 30.15 last 15.43)
 - mkdir 568 (time 1787728858.03 total 40.16 last 13.89)
 - mkdir 699 (time 1787728868.03 total 50.16 last 13.10)
 - mkdir 846 (time 1787728878.11 total 60.24 last 14.59)
 - mkdir 974 (time 1787728888.31 total 70.45 last 12.54)
total: 1000 mkdir in 74.56 seconds: 13.41 ops/second
 - mkdir 130 (time 1787728827.47 total 10.04 last 12.95)
 - mkdir 271 (time 1787728837.56 total 20.13 last 13.98)
 - mkdir 418 (time 1787728847.59 total 30.16 last 14.65)
 - mkdir 558 (time 1787728857.64 total 40.21 last 13.93)
 - mkdir 693 (time 1787728867.73 total 50.30 last 13.38)
 - mkdir 841 (time 1787728877.77 total 60.34 last 14.74)
 - mkdir 966 (time 1787728887.96 total 70.53 last 12.27)
total: 1000 mkdir in 75.65 seconds: 13.22 ops/second
 - mkdir 127 (time 1787728827.02 total 10.01 last 12.68)
 - mkdir 263 (time 1787728837.08 total 20.07 last 13.52)
 - mkdir 396 (time 1787728847.14 total 30.14 last 13.22)
 - mkdir 526 (time 1787728857.15 total 40.14 last 13.00)
 - mkdir 653 (time 1787728867.15 total 50.14 last 12.70)
 - mkdir 791 (time 1787728877.16 total 60.15 last 13.79)
 - mkdir 915 (time 1787728887.34 total 70.33 last 12.19)
total: 1000 mkdir in 79.22 seconds: 12.62 ops/second
Time for file creation 90 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 16 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.47 seconds: 21.08 ops/second
 - mkdir 148 (time 1787729480.91 total 10.04 last 14.74)
 - mkdir 285 (time 1787729490.93 total 20.07 last 13.66)
 - mkdir 412 (time 1787729500.97 total 30.10 last 12.65)
 - mkdir 563 (time 1787729511.01 total 40.15 last 15.03)
 - mkdir 683 (time 1787729521.02 total 50.15 last 12.00)
 - mkdir 832 (time 1787729531.06 total 60.19 last 14.84)
 - mkdir 993 (time 1787729541.14 total 70.27 last 15.97)
total: 1000 mkdir in 70.80 seconds: 14.12 ops/second
 - mkdir 152 (time 1787729482.58 total 10.05 last 15.13)
 - mkdir 304 (time 1787729492.65 total 20.12 last 15.09)
 - mkdir 436 (time 1787729502.69 total 30.16 last 13.15)
 - mkdir 584 (time 1787729512.77 total 40.24 last 14.69)
 - mkdir 711 (time 1787729522.92 total 50.39 last 12.50)
 - mkdir 862 (time 1787729532.97 total 60.43 last 15.04)
total: 1000 mkdir in 69.55 seconds: 14.38 ops/second
 - mkdir 139 (time 1787729480.46 total 10.04 last 13.84)
 - mkdir 281 (time 1787729490.51 total 20.09 last 14.14)
 - mkdir 408 (time 1787729500.52 total 30.10 last 12.68)
 - mkdir 557 (time 1787729510.53 total 40.11 last 14.89)
 - mkdir 674 (time 1787729520.55 total 50.13 last 11.68)
 - mkdir 810 (time 1787729530.57 total 60.15 last 13.57)
 - mkdir 973 (time 1787729540.60 total 70.18 last 16.24)
total: 1000 mkdir in 72.42 seconds: 13.81 ops/second
 - mkdir 148 (time 1787729482.94 total 10.02 last 14.77)
 - mkdir 295 (time 1787729492.96 total 20.04 last 14.67)
 - mkdir 424 (time 1787729502.97 total 30.05 last 12.88)
 - mkdir 570 (time 1787729512.99 total 40.07 last 14.58)
 - mkdir 701 (time 1787729523.03 total 50.12 last 13.04)
 - mkdir 855 (time 1787729533.10 total 60.19 last 15.29)
 - mkdir 996 (time 1787729543.14 total 70.22 last 14.05)
total: 1000 mkdir in 70.59 seconds: 14.17 ops/second
 - mkdir 145 (time 1787729481.79 total 10.01 last 14.48)
 - mkdir 289 (time 1787729491.84 total 20.06 last 14.33)
 - mkdir 414 (time 1787729501.92 total 30.15 last 12.40)
 - mkdir 564 (time 1787729511.92 total 40.15 last 14.99)
 - mkdir 684 (time 1787729521.94 total 50.17 last 11.98)
 - mkdir 833 (time 1787729532.01 total 60.23 last 14.81)
 - mkdir 985 (time 1787729542.08 total 70.30 last 15.09)
total: 1000 mkdir in 71.84 seconds: 13.92 ops/second
 - mkdir 143 (time 1787729482.29 total 10.02 last 14.27)
 - mkdir 280 (time 1787729492.30 total 20.03 last 13.69)
 - mkdir 406 (time 1787729502.36 total 30.09 last 12.52)
 - mkdir 554 (time 1787729512.39 total 40.12 last 14.76)
 - mkdir 676 (time 1787729522.42 total 50.15 last 12.17)
 - mkdir 831 (time 1787729532.44 total 60.17 last 15.47)
 - mkdir 978 (time 1787729542.50 total 70.23 last 14.61)
total: 1000 mkdir in 72.37 seconds: 13.82 ops/second
 - mkdir 143 (time 1787729481.88 total 10.04 last 14.24)
 - mkdir 278 (time 1787729491.88 total 20.05 last 13.49)
 - mkdir 406 (time 1787729501.92 total 30.09 last 12.75)
 - mkdir 553 (time 1787729511.96 total 40.12 last 14.65)
 - mkdir 674 (time 1787729522.05 total 50.21 last 12.00)
 - mkdir 817 (time 1787729532.09 total 60.25 last 14.24)
 - mkdir 970 (time 1787729542.12 total 70.28 last 15.26)
total: 1000 mkdir in 73.34 seconds: 13.63 ops/second
 - mkdir 145 (time 1787729482.47 total 10.06 last 14.41)
 - mkdir 285 (time 1787729492.50 total 20.09 last 13.96)
 - mkdir 411 (time 1787729502.54 total 30.13 last 12.55)
 - mkdir 557 (time 1787729512.58 total 40.17 last 14.54)
 - mkdir 678 (time 1787729522.61 total 50.21 last 12.06)
 - mkdir 829 (time 1787729532.66 total 60.26 last 15.02)
 - mkdir 974 (time 1787729542.71 total 70.30 last 14.44)
total: 1000 mkdir in 73.08 seconds: 13.68 ops/second
 - mkdir 136 (time 1787729480.95 total 10.01 last 13.59)
 - mkdir 270 (time 1787729490.98 total 20.04 last 13.36)
 - mkdir 393 (time 1787729501.03 total 30.09 last 12.24)
 - mkdir 539 (time 1787729511.09 total 40.16 last 14.50)
 - mkdir 651 (time 1787729521.13 total 50.19 last 11.16)
 - mkdir 786 (time 1787729531.19 total 60.25 last 13.42)
 - mkdir 940 (time 1787729541.27 total 70.33 last 15.27)
total: 1000 mkdir in 76.49 seconds: 13.07 ops/second
 - mkdir 142 (time 1787729482.71 total 10.03 last 14.16)
 - mkdir 281 (time 1787729492.72 total 20.03 last 13.89)
 - mkdir 404 (time 1787729502.81 total 30.13 last 12.18)
 - mkdir 543 (time 1787729512.88 total 40.20 last 13.80)
 - mkdir 662 (time 1787729522.93 total 50.24 last 11.85)
 - mkdir 813 (time 1787729532.98 total 60.30 last 15.02)
 - mkdir 955 (time 1787729543.01 total 70.32 last 14.16)
total: 1000 mkdir in 74.88 seconds: 13.36 ops/second
Time for file creation 88 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787729556 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787729557 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787729557 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787729557 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787729557 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787729557 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787729557 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787729557 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787729557 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787729558 ; total 1 ; last 1)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
Time for file removal 46 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8