== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 17:25:18 (1789075518)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.60 seconds: 16.73 ops/second
 - mkdir 149 (time 1789075541.67 total 10.04 last 14.84)
 - mkdir 312 (time 1789075551.68 total 20.05 last 16.29)
 - mkdir 468 (time 1789075561.73 total 30.10 last 15.51)
 - mkdir 621 (time 1789075571.75 total 40.12 last 15.27)
 - mkdir 779 (time 1789075581.80 total 50.17 last 15.73)
 - mkdir 931 (time 1789075591.87 total 60.23 last 15.10)
total: 1000 mkdir in 64.96 seconds: 15.39 ops/second
 - mkdir 156 (time 1789075542.95 total 10.04 last 15.53)
 - mkdir 323 (time 1789075553.01 total 20.11 last 16.59)
 - mkdir 485 (time 1789075563.07 total 30.17 last 16.11)
 - mkdir 634 (time 1789075573.09 total 40.18 last 14.87)
 - mkdir 788 (time 1789075583.10 total 50.20 last 15.38)
 - mkdir 938 (time 1789075593.16 total 60.25 last 14.91)
total: 1000 mkdir in 65.35 seconds: 15.30 ops/second
 - mkdir 127 (time 1789075539.01 total 10.01 last 12.68)
 - mkdir 270 (time 1789075549.04 total 20.04 last 14.26)
 - mkdir 422 (time 1789075559.12 total 30.12 last 15.08)
 - mkdir 586 (time 1789075569.20 total 40.20 last 16.27)
 - mkdir 721 (time 1789075579.22 total 50.22 last 13.48)
 - mkdir 881 (time 1789075589.29 total 60.29 last 15.89)
total: 1000 mkdir in 70.04 seconds: 14.28 ops/second
 - mkdir 147 (time 1789075542.88 total 10.01 last 14.68)
 - mkdir 309 (time 1789075552.88 total 20.02 last 16.19)
 - mkdir 463 (time 1789075562.93 total 30.07 last 15.32)
 - mkdir 614 (time 1789075572.96 total 40.10 last 15.06)
 - mkdir 774 (time 1789075583.00 total 50.13 last 15.94)
 - mkdir 921 (time 1789075593.06 total 60.19 last 14.61)
total: 1000 mkdir in 66.92 seconds: 14.94 ops/second
 - mkdir 146 (time 1789075541.73 total 10.03 last 14.56)
 - mkdir 300 (time 1789075551.74 total 20.04 last 15.38)
 - mkdir 452 (time 1789075561.82 total 30.11 last 15.09)
 - mkdir 604 (time 1789075571.82 total 40.11 last 15.20)
 - mkdir 757 (time 1789075581.84 total 50.13 last 15.27)
 - mkdir 909 (time 1789075591.87 total 60.17 last 15.15)
total: 1000 mkdir in 68.22 seconds: 14.66 ops/second
 - mkdir 148 (time 1789075542.03 total 10.02 last 14.78)
 - mkdir 300 (time 1789075552.07 total 20.06 last 15.14)
 - mkdir 455 (time 1789075562.11 total 30.09 last 15.45)
 - mkdir 606 (time 1789075572.14 total 40.12 last 15.06)
 - mkdir 761 (time 1789075582.14 total 50.13 last 15.49)
 - mkdir 912 (time 1789075592.18 total 60.17 last 15.04)
total: 1000 mkdir in 67.92 seconds: 14.72 ops/second
 - mkdir 152 (time 1789075543.38 total 10.02 last 15.17)
 - mkdir 312 (time 1789075553.47 total 20.10 last 15.86)
 - mkdir 469 (time 1789075563.48 total 30.11 last 15.68)
 - mkdir 619 (time 1789075573.53 total 40.16 last 14.93)
 - mkdir 772 (time 1789075583.57 total 50.21 last 15.23)
 - mkdir 924 (time 1789075593.58 total 60.22 last 15.18)
total: 1000 mkdir in 66.98 seconds: 14.93 ops/second
 - mkdir 149 (time 1789075543.14 total 10.08 last 14.78)
 - mkdir 307 (time 1789075553.18 total 20.12 last 15.74)
 - mkdir 468 (time 1789075563.22 total 30.16 last 16.03)
 - mkdir 616 (time 1789075573.24 total 40.18 last 14.78)
 - mkdir 769 (time 1789075583.32 total 50.25 last 15.18)
 - mkdir 921 (time 1789075593.35 total 60.29 last 15.15)
total: 1000 mkdir in 67.39 seconds: 14.84 ops/second
 - mkdir 146 (time 1789075542.98 total 10.07 last 14.50)
 - mkdir 303 (time 1789075553.00 total 20.09 last 15.68)
 - mkdir 462 (time 1789075563.11 total 30.20 last 15.72)
 - mkdir 612 (time 1789075573.11 total 40.20 last 14.99)
 - mkdir 769 (time 1789075583.14 total 50.23 last 15.65)
 - mkdir 915 (time 1789075593.19 total 60.28 last 14.54)
total: 1000 mkdir in 67.76 seconds: 14.76 ops/second
 - mkdir 146 (time 1789075542.57 total 10.04 last 14.55)
 - mkdir 301 (time 1789075552.61 total 20.07 last 15.45)
 - mkdir 457 (time 1789075562.70 total 30.16 last 15.46)
 - mkdir 608 (time 1789075572.76 total 40.22 last 15.00)
 - mkdir 762 (time 1789075582.80 total 50.26 last 15.35)
 - mkdir 911 (time 1789075592.85 total 60.31 last 14.83)
total: 1000 mkdir in 68.41 seconds: 14.62 ops/second
Time for file creation 82 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 15 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.56 seconds: 17.98 ops/second
 - mkdir 131 (time 1789076224.53 total 10.07 last 13.00)
 - mkdir 284 (time 1789076234.56 total 20.10 last 15.26)
 - mkdir 400 (time 1789076244.59 total 30.13 last 11.57)
 - mkdir 530 (time 1789076254.64 total 40.18 last 12.94)
 - mkdir 629 (time 1789076264.69 total 50.23 last 9.85)
 - mkdir 757 (time 1789076274.70 total 60.24 last 12.79)
 - mkdir 911 (time 1789076284.70 total 70.24 last 15.39)
total: 1000 mkdir in 76.36 seconds: 13.10 ops/second
 - mkdir 116 (time 1789076222.95 total 10.09 last 11.50)
 - mkdir 275 (time 1789076233.03 total 20.17 last 15.77)
 - mkdir 395 (time 1789076243.09 total 30.23 last 11.92)
 - mkdir 516 (time 1789076253.13 total 40.27 last 12.06)
 - mkdir 619 (time 1789076263.22 total 50.36 last 10.21)
 - mkdir 740 (time 1789076273.29 total 60.43 last 12.02)
 - mkdir 889 (time 1789076283.32 total 70.47 last 14.85)
total: 1000 mkdir in 77.97 seconds: 12.83 ops/second
 - mkdir 125 (time 1789076223.31 total 10.08 last 12.40)
 - mkdir 276 (time 1789076233.34 total 20.11 last 15.05)
 - mkdir 394 (time 1789076243.47 total 30.24 last 11.65)
 - mkdir 519 (time 1789076253.53 total 40.30 last 12.42)
 - mkdir 623 (time 1789076263.64 total 50.41 last 10.28)
 - mkdir 736 (time 1789076273.67 total 60.44 last 11.27)
 - mkdir 890 (time 1789076283.69 total 70.46 last 15.37)
total: 1000 mkdir in 77.95 seconds: 12.83 ops/second
 - mkdir 131 (time 1789076224.79 total 10.04 last 13.05)
 - mkdir 286 (time 1789076234.85 total 20.10 last 15.41)
 - mkdir 405 (time 1789076244.88 total 30.13 last 11.86)
 - mkdir 530 (time 1789076255.03 total 40.28 last 12.31)
 - mkdir 631 (time 1789076265.09 total 50.34 last 10.04)
 - mkdir 756 (time 1789076275.19 total 60.44 last 12.37)
 - mkdir 916 (time 1789076285.25 total 70.50 last 15.90)
total: 1000 mkdir in 76.96 seconds: 12.99 ops/second
 - mkdir 129 (time 1789076224.91 total 10.05 last 12.84)
 - mkdir 280 (time 1789076235.07 total 20.21 last 14.86)
 - mkdir 400 (time 1789076245.09 total 30.23 last 11.98)
 - mkdir 527 (time 1789076255.22 total 40.36 last 12.54)
 - mkdir 628 (time 1789076265.34 total 50.48 last 9.98)
 - mkdir 751 (time 1789076275.37 total 60.51 last 12.27)
 - mkdir 908 (time 1789076285.44 total 70.58 last 15.59)
total: 1000 mkdir in 77.93 seconds: 12.83 ops/second
 - mkdir 122 (time 1789076224.31 total 10.01 last 12.19)
 - mkdir 274 (time 1789076234.32 total 20.02 last 15.19)
 - mkdir 390 (time 1789076244.39 total 30.09 last 11.52)
 - mkdir 519 (time 1789076254.44 total 40.13 last 12.84)
 - mkdir 622 (time 1789076264.62 total 50.31 last 10.12)
 - mkdir 751 (time 1789076274.67 total 60.36 last 12.84)
 - mkdir 902 (time 1789076284.71 total 70.40 last 15.04)
total: 1000 mkdir in 79.03 seconds: 12.65 ops/second
 - mkdir 128 (time 1789076225.46 total 10.00 last 12.80)
 - mkdir 268 (time 1789076235.62 total 20.16 last 13.78)
 - mkdir 384 (time 1789076245.67 total 30.22 last 11.54)
 - mkdir 508 (time 1789076255.72 total 40.27 last 12.34)
 - mkdir 612 (time 1789076265.76 total 50.31 last 10.36)
 - mkdir 740 (time 1789076275.83 total 60.37 last 12.72)
 - mkdir 899 (time 1789076285.83 total 70.38 last 15.89)
total: 1000 mkdir in 80.13 seconds: 12.48 ops/second
 - mkdir 128 (time 1789076225.48 total 10.03 last 12.76)
 - mkdir 273 (time 1789076235.71 total 20.26 last 14.18)
 - mkdir 388 (time 1789076245.80 total 30.34 last 11.40)
 - mkdir 513 (time 1789076255.89 total 40.44 last 12.39)
 - mkdir 621 (time 1789076265.95 total 50.50 last 10.73)
 - mkdir 745 (time 1789076275.96 total 60.51 last 12.39)
 - mkdir 892 (time 1789076286.02 total 70.57 last 14.62)
 - mkdir 997 (time 1789076296.06 total 80.61 last 10.46)
total: 1000 mkdir in 80.80 seconds: 12.38 ops/second
 - mkdir 126 (time 1789076225.33 total 10.08 last 12.50)
 - mkdir 268 (time 1789076235.41 total 20.16 last 14.09)
 - mkdir 382 (time 1789076245.48 total 30.24 last 11.32)
 - mkdir 505 (time 1789076255.48 total 40.24 last 12.29)
 - mkdir 606 (time 1789076265.51 total 50.26 last 10.08)
 - mkdir 730 (time 1789076275.53 total 60.29 last 12.37)
 - mkdir 885 (time 1789076285.56 total 70.32 last 15.45)
 - mkdir 990 (time 1789076295.61 total 80.36 last 10.46)
total: 1000 mkdir in 81.67 seconds: 12.24 ops/second
 - mkdir 122 (time 1789076224.69 total 10.03 last 12.16)
 - mkdir 267 (time 1789076234.73 total 20.07 last 14.45)
 - mkdir 373 (time 1789076244.76 total 30.10 last 10.57)
 - mkdir 499 (time 1789076254.76 total 40.11 last 12.59)
 - mkdir 598 (time 1789076264.80 total 50.14 last 9.87)
 - mkdir 725 (time 1789076274.80 total 60.15 last 12.69)
 - mkdir 871 (time 1789076284.82 total 70.16 last 14.58)
 - mkdir 976 (time 1789076294.82 total 80.16 last 10.50)
total: 1000 mkdir in 83.38 seconds: 11.99 ops/second
Time for file creation 96 sec for 1 RPC sent at a time
 - unlinked 0 (time 1789076310 ; total 1 ; last 1)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1789076311 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1789076311 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1789076312 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1789076312 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1789076313 ; total 0 ; last 0)
total: 1000 unlinks in 40 seconds: 25.000000 unlinks/second
 - unlinked 0 (time 1789076313 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1789076313 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1789076312 ; total 0 ; last 0)
total: 1000 unlinks in 42 seconds: 23.809525 unlinks/second
 - unlinked 0 (time 1789076313 ; total 0 ; last 0)
total: 1000 unlinks in 42 seconds: 23.809525 unlinks/second
Time for file removal 58 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8