== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 01:31:31 (1788672691)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.52 seconds: 19.28 ops/second
 - mkdir 243 (time 1788672709.92 total 10.05 last 24.18)
 - mkdir 471 (time 1788672719.97 total 20.10 last 22.69)
 - mkdir 705 (time 1788672730.03 total 30.17 last 23.24)
 - mkdir 891 (time 1788672740.10 total 40.24 last 18.47)
total: 1000 mkdir in 46.25 seconds: 21.62 ops/second
 - mkdir 253 (time 1788672710.86 total 10.04 last 25.21)
 - mkdir 483 (time 1788672720.89 total 20.07 last 22.92)
 - mkdir 716 (time 1788672730.97 total 30.15 last 23.12)
 - mkdir 898 (time 1788672740.98 total 40.16 last 18.18)
total: 1000 mkdir in 46.34 seconds: 21.58 ops/second
 - mkdir 248 (time 1788672710.29 total 10.02 last 24.76)
 - mkdir 469 (time 1788672720.31 total 20.04 last 22.06)
 - mkdir 709 (time 1788672730.34 total 30.07 last 23.91)
 - mkdir 889 (time 1788672740.37 total 40.10 last 17.95)
total: 1000 mkdir in 47.32 seconds: 21.13 ops/second
 - mkdir 247 (time 1788672710.35 total 10.01 last 24.67)
 - mkdir 471 (time 1788672720.36 total 20.01 last 22.39)
 - mkdir 705 (time 1788672730.44 total 30.10 last 23.20)
 - mkdir 887 (time 1788672740.45 total 40.11 last 18.19)
total: 1000 mkdir in 47.43 seconds: 21.08 ops/second
 - mkdir 253 (time 1788672710.75 total 10.01 last 25.28)
 - mkdir 478 (time 1788672720.77 total 20.02 last 22.46)
 - mkdir 708 (time 1788672730.81 total 30.06 last 22.91)
 - mkdir 889 (time 1788672740.81 total 40.07 last 18.09)
total: 1000 mkdir in 47.15 seconds: 21.21 ops/second
 - mkdir 260 (time 1788672710.98 total 10.01 last 25.97)
 - mkdir 487 (time 1788672720.98 total 20.01 last 22.70)
 - mkdir 714 (time 1788672731.02 total 30.05 last 22.62)
 - mkdir 898 (time 1788672741.12 total 40.15 last 18.22)
total: 1000 mkdir in 47.00 seconds: 21.28 ops/second
 - mkdir 252 (time 1788672710.63 total 10.02 last 25.15)
 - mkdir 478 (time 1788672720.64 total 20.03 last 22.58)
 - mkdir 711 (time 1788672730.67 total 30.05 last 23.24)
 - mkdir 889 (time 1788672740.67 total 40.06 last 17.79)
total: 1000 mkdir in 47.42 seconds: 21.09 ops/second
 - mkdir 257 (time 1788672711.07 total 10.01 last 25.68)
 - mkdir 480 (time 1788672721.09 total 20.02 last 22.28)
 - mkdir 704 (time 1788672731.11 total 30.05 last 22.34)
 - mkdir 887 (time 1788672741.16 total 40.10 last 18.21)
total: 1000 mkdir in 47.52 seconds: 21.04 ops/second
 - mkdir 237 (time 1788672710.29 total 10.03 last 23.63)
 - mkdir 454 (time 1788672720.30 total 20.03 last 21.69)
 - mkdir 687 (time 1788672730.34 total 30.07 last 23.20)
 - mkdir 868 (time 1788672740.36 total 40.09 last 18.07)
total: 1000 mkdir in 49.18 seconds: 20.33 ops/second
 - mkdir 248 (time 1788672710.68 total 10.00 last 24.79)
 - mkdir 463 (time 1788672720.69 total 20.01 last 21.48)
 - mkdir 693 (time 1788672730.74 total 30.07 last 22.87)
 - mkdir 867 (time 1788672740.76 total 40.09 last 17.37)
total: 1000 mkdir in 49.62 seconds: 20.15 ops/second
Time for file creation 58 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 11 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.44 seconds: 22.75 ops/second
 - mkdir 186 (time 1788673280.74 total 10.02 last 18.56)
 - mkdir 346 (time 1788673290.84 total 20.12 last 15.85)
 - mkdir 543 (time 1788673300.85 total 30.13 last 19.68)
 - mkdir 716 (time 1788673310.86 total 40.15 last 17.28)
 - mkdir 910 (time 1788673320.95 total 50.23 last 19.23)
total: 1000 mkdir in 56.15 seconds: 17.81 ops/second
 - mkdir 181 (time 1788673280.78 total 10.00 last 18.10)
 - mkdir 335 (time 1788673290.81 total 20.03 last 15.36)
 - mkdir 529 (time 1788673300.86 total 30.08 last 19.30)
 - mkdir 693 (time 1788673310.89 total 40.11 last 16.35)
 - mkdir 887 (time 1788673320.96 total 50.18 last 19.27)
total: 1000 mkdir in 58.42 seconds: 17.12 ops/second
 - mkdir 182 (time 1788673281.65 total 10.02 last 18.16)
 - mkdir 342 (time 1788673291.67 total 20.05 last 15.96)
 - mkdir 534 (time 1788673301.69 total 30.06 last 19.17)
 - mkdir 708 (time 1788673311.73 total 40.11 last 17.32)
 - mkdir 897 (time 1788673321.75 total 50.12 last 18.88)
total: 1000 mkdir in 58.75 seconds: 17.02 ops/second
 - mkdir 177 (time 1788673280.66 total 10.00 last 17.69)
 - mkdir 336 (time 1788673290.69 total 20.03 last 15.85)
 - mkdir 526 (time 1788673300.70 total 30.04 last 18.98)
 - mkdir 692 (time 1788673310.70 total 40.05 last 16.59)
 - mkdir 879 (time 1788673320.72 total 50.06 last 18.68)
total: 1000 mkdir in 59.82 seconds: 16.72 ops/second
 - mkdir 182 (time 1788673281.18 total 10.06 last 18.09)
 - mkdir 339 (time 1788673291.22 total 20.10 last 15.64)
 - mkdir 541 (time 1788673301.26 total 30.14 last 20.11)
 - mkdir 701 (time 1788673311.31 total 40.19 last 15.92)
 - mkdir 884 (time 1788673321.35 total 50.23 last 18.23)
total: 1000 mkdir in 60.07 seconds: 16.65 ops/second
 - mkdir 172 (time 1788673280.36 total 10.00 last 17.19)
 - mkdir 327 (time 1788673290.38 total 20.03 last 15.47)
 - mkdir 515 (time 1788673300.47 total 30.11 last 18.64)
 - mkdir 683 (time 1788673310.48 total 40.12 last 16.79)
 - mkdir 871 (time 1788673320.48 total 50.12 last 18.80)
 - mkdir 993 (time 1788673330.53 total 60.17 last 12.13)
total: 1000 mkdir in 61.14 seconds: 16.36 ops/second
 - mkdir 178 (time 1788673281.22 total 10.03 last 17.74)
 - mkdir 334 (time 1788673291.27 total 20.08 last 15.53)
 - mkdir 527 (time 1788673301.30 total 30.11 last 19.25)
 - mkdir 692 (time 1788673311.37 total 40.18 last 16.38)
 - mkdir 878 (time 1788673321.42 total 50.23 last 18.52)
 - mkdir 997 (time 1788673331.47 total 60.28 last 11.84)
total: 1000 mkdir in 60.50 seconds: 16.53 ops/second
 - mkdir 176 (time 1788673281.58 total 10.09 last 17.44)
 - mkdir 331 (time 1788673291.62 total 20.13 last 15.45)
 - mkdir 529 (time 1788673301.64 total 30.15 last 19.75)
 - mkdir 693 (time 1788673311.65 total 40.16 last 16.38)
 - mkdir 883 (time 1788673321.70 total 50.21 last 18.90)
 - mkdir 995 (time 1788673331.72 total 60.24 last 11.17)
total: 1000 mkdir in 60.73 seconds: 16.47 ops/second
 - mkdir 177 (time 1788673281.03 total 10.07 last 17.59)
 - mkdir 329 (time 1788673291.05 total 20.09 last 15.16)
 - mkdir 525 (time 1788673301.13 total 30.16 last 19.46)
 - mkdir 683 (time 1788673311.20 total 40.24 last 15.69)
 - mkdir 869 (time 1788673321.23 total 50.26 last 18.55)
 - mkdir 989 (time 1788673331.29 total 60.33 last 11.92)
total: 1000 mkdir in 61.43 seconds: 16.28 ops/second
 - mkdir 164 (time 1788673280.02 total 10.07 last 16.29)
 - mkdir 317 (time 1788673290.11 total 20.16 last 15.17)
 - mkdir 499 (time 1788673300.15 total 30.20 last 18.12)
 - mkdir 660 (time 1788673310.18 total 40.23 last 16.06)
 - mkdir 840 (time 1788673320.21 total 50.26 last 17.94)
 - mkdir 963 (time 1788673330.23 total 60.28 last 12.27)
total: 1000 mkdir in 63.69 seconds: 15.70 ops/second
Time for file creation 74 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788673340 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788673341 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788673341 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788673342 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788673342 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788673342 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788673341 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788673342 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788673342 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788673342 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
Time for file removal 41 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8