== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 06:18:41 (1788862721)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.45 seconds: 22.33 ops/second
 - mkdir 185 (time 1788862741.94 total 10.04 last 18.44)
 - mkdir 350 (time 1788862751.99 total 20.08 last 16.42)
 - mkdir 494 (time 1788862761.99 total 30.08 last 14.40)
 - mkdir 613 (time 1788862772.00 total 40.10 last 11.88)
 - mkdir 781 (time 1788862782.04 total 50.14 last 16.74)
 - mkdir 951 (time 1788862792.05 total 60.15 last 16.98)
total: 1000 mkdir in 62.99 seconds: 15.87 ops/second
 - mkdir 179 (time 1788862741.87 total 10.02 last 17.87)
 - mkdir 344 (time 1788862751.91 total 20.06 last 16.43)
 - mkdir 479 (time 1788862761.94 total 30.09 last 13.46)
 - mkdir 592 (time 1788862771.96 total 40.11 last 11.28)
 - mkdir 756 (time 1788862781.96 total 50.11 last 16.40)
 - mkdir 928 (time 1788862791.97 total 60.12 last 17.19)
total: 1000 mkdir in 65.13 seconds: 15.35 ops/second
 - mkdir 177 (time 1788862743.43 total 10.08 last 17.56)
 - mkdir 338 (time 1788862753.46 total 20.10 last 16.06)
 - mkdir 476 (time 1788862763.46 total 30.11 last 13.80)
 - mkdir 592 (time 1788862773.51 total 40.15 last 11.55)
 - mkdir 766 (time 1788862783.57 total 50.22 last 17.29)
 - mkdir 934 (time 1788862793.59 total 60.23 last 16.77)
total: 1000 mkdir in 66.70 seconds: 14.99 ops/second
 - mkdir 171 (time 1788862742.88 total 10.08 last 16.97)
 - mkdir 335 (time 1788862752.96 total 20.15 last 16.29)
 - mkdir 472 (time 1788862762.98 total 30.17 last 13.67)
 - mkdir 585 (time 1788862773.00 total 40.20 last 11.27)
 - mkdir 756 (time 1788862783.06 total 50.25 last 17.01)
 - mkdir 926 (time 1788862793.11 total 60.30 last 16.91)
total: 1000 mkdir in 67.32 seconds: 14.85 ops/second
 - mkdir 177 (time 1788862743.68 total 10.07 last 17.57)
 - mkdir 339 (time 1788862753.73 total 20.12 last 16.12)
 - mkdir 468 (time 1788862763.82 total 30.22 last 12.78)
 - mkdir 588 (time 1788862773.86 total 40.25 last 11.96)
 - mkdir 757 (time 1788862783.88 total 50.27 last 16.87)
 - mkdir 927 (time 1788862793.88 total 60.28 last 16.99)
total: 1000 mkdir in 68.31 seconds: 14.64 ops/second
 - mkdir 171 (time 1788862743.77 total 10.03 last 17.05)
 - mkdir 338 (time 1788862753.86 total 20.12 last 16.55)
 - mkdir 472 (time 1788862763.89 total 30.15 last 13.35)
 - mkdir 586 (time 1788862773.91 total 40.16 last 11.39)
 - mkdir 755 (time 1788862783.95 total 50.21 last 16.82)
 - mkdir 922 (time 1788862793.95 total 60.21 last 16.69)
total: 1000 mkdir in 68.55 seconds: 14.59 ops/second
 - mkdir 175 (time 1788862743.55 total 10.07 last 17.38)
 - mkdir 344 (time 1788862753.63 total 20.14 last 16.77)
 - mkdir 477 (time 1788862763.65 total 30.16 last 13.28)
 - mkdir 589 (time 1788862773.65 total 40.16 last 11.20)
 - mkdir 757 (time 1788862783.70 total 50.21 last 16.71)
 - mkdir 921 (time 1788862793.71 total 60.22 last 16.39)
total: 1000 mkdir in 68.97 seconds: 14.50 ops/second
 - mkdir 170 (time 1788862742.37 total 10.02 last 16.96)
 - mkdir 324 (time 1788862752.39 total 20.04 last 15.37)
 - mkdir 451 (time 1788862762.40 total 30.05 last 12.69)
 - mkdir 566 (time 1788862772.45 total 40.10 last 11.44)
 - mkdir 724 (time 1788862782.48 total 50.13 last 15.77)
 - mkdir 893 (time 1788862792.53 total 60.18 last 16.81)
 - mkdir 993 (time 1788862802.60 total 70.25 last 9.93)
total: 1000 mkdir in 70.95 seconds: 14.09 ops/second
 - mkdir 169 (time 1788862743.75 total 10.06 last 16.80)
 - mkdir 329 (time 1788862753.86 total 20.17 last 15.81)
 - mkdir 458 (time 1788862763.87 total 30.18 last 12.89)
 - mkdir 571 (time 1788862773.89 total 40.20 last 11.28)
 - mkdir 736 (time 1788862783.89 total 50.20 last 16.49)
 - mkdir 905 (time 1788862793.92 total 60.23 last 16.85)
total: 1000 mkdir in 70.04 seconds: 14.28 ops/second
 - mkdir 165 (time 1788862743.43 total 10.03 last 16.45)
 - mkdir 322 (time 1788862753.46 total 20.07 last 15.64)
 - mkdir 449 (time 1788862763.50 total 30.10 last 12.66)
 - mkdir 554 (time 1788862773.55 total 40.15 last 10.45)
 - mkdir 709 (time 1788862783.63 total 50.24 last 15.37)
 - mkdir 869 (time 1788862793.66 total 60.26 last 15.96)
 - mkdir 960 (time 1788862803.68 total 70.29 last 9.08)
total: 1000 mkdir in 72.66 seconds: 13.76 ops/second
Time for file creation 82 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 10 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.52 seconds: 19.31 ops/second
 - mkdir 127 (time 1788863357.72 total 10.04 last 12.65)
 - mkdir 270 (time 1788863367.75 total 20.07 last 14.26)
 - mkdir 414 (time 1788863377.75 total 30.07 last 14.40)
 - mkdir 562 (time 1788863387.77 total 40.09 last 14.77)
 - mkdir 728 (time 1788863397.80 total 50.12 last 16.55)
 - mkdir 896 (time 1788863407.82 total 60.14 last 16.76)
total: 1000 mkdir in 65.93 seconds: 15.17 ops/second
 - mkdir 137 (time 1788863361.62 total 10.05 last 13.63)
 - mkdir 292 (time 1788863371.66 total 20.09 last 15.44)
 - mkdir 435 (time 1788863381.72 total 30.15 last 14.21)
 - mkdir 601 (time 1788863391.75 total 40.18 last 16.56)
 - mkdir 772 (time 1788863401.75 total 50.18 last 17.09)
 - mkdir 948 (time 1788863411.77 total 60.21 last 17.56)
total: 1000 mkdir in 64.11 seconds: 15.60 ops/second
 - mkdir 136 (time 1788863361.61 total 10.03 last 13.56)
 - mkdir 286 (time 1788863371.66 total 20.08 last 14.92)
 - mkdir 429 (time 1788863381.74 total 30.16 last 14.20)
 - mkdir 592 (time 1788863391.77 total 40.19 last 16.24)
 - mkdir 763 (time 1788863401.81 total 50.23 last 17.03)
 - mkdir 936 (time 1788863411.81 total 60.23 last 17.30)
total: 1000 mkdir in 65.91 seconds: 15.17 ops/second
 - mkdir 132 (time 1788863361.56 total 10.00 last 13.19)
 - mkdir 280 (time 1788863371.58 total 20.03 last 14.77)
 - mkdir 422 (time 1788863381.67 total 30.12 last 14.07)
 - mkdir 590 (time 1788863391.67 total 40.12 last 16.79)
 - mkdir 759 (time 1788863401.70 total 50.15 last 16.85)
 - mkdir 933 (time 1788863411.75 total 60.20 last 17.31)
total: 1000 mkdir in 66.01 seconds: 15.15 ops/second
 - mkdir 136 (time 1788863361.91 total 10.03 last 13.56)
 - mkdir 286 (time 1788863372.03 total 20.15 last 14.82)
 - mkdir 427 (time 1788863382.05 total 30.17 last 14.07)
 - mkdir 593 (time 1788863392.07 total 40.19 last 16.58)
 - mkdir 766 (time 1788863402.11 total 50.23 last 17.23)
 - mkdir 937 (time 1788863412.15 total 60.27 last 17.03)
total: 1000 mkdir in 66.37 seconds: 15.07 ops/second
 - mkdir 135 (time 1788863362.06 total 10.00 last 13.49)
 - mkdir 286 (time 1788863372.09 total 20.04 last 15.05)
 - mkdir 426 (time 1788863382.14 total 30.09 last 13.92)
 - mkdir 587 (time 1788863392.16 total 40.11 last 16.07)
 - mkdir 761 (time 1788863402.23 total 50.18 last 17.28)
 - mkdir 932 (time 1788863412.25 total 60.20 last 17.06)
total: 1000 mkdir in 67.23 seconds: 14.87 ops/second
 - mkdir 132 (time 1788863361.88 total 10.03 last 13.16)
 - mkdir 280 (time 1788863371.96 total 20.11 last 14.69)
 - mkdir 422 (time 1788863381.97 total 30.11 last 14.19)
 - mkdir 583 (time 1788863392.05 total 40.20 last 15.96)
 - mkdir 753 (time 1788863402.10 total 50.24 last 16.93)
 - mkdir 924 (time 1788863412.11 total 60.26 last 17.07)
total: 1000 mkdir in 67.95 seconds: 14.72 ops/second
 - mkdir 133 (time 1788863361.09 total 10.15 last 13.10)
 - mkdir 267 (time 1788863371.14 total 20.20 last 13.34)
 - mkdir 404 (time 1788863381.20 total 30.25 last 13.62)
 - mkdir 559 (time 1788863391.20 total 40.25 last 15.50)
 - mkdir 732 (time 1788863401.24 total 50.29 last 17.23)
 - mkdir 907 (time 1788863411.26 total 60.31 last 17.47)
total: 1000 mkdir in 69.06 seconds: 14.48 ops/second
 - mkdir 134 (time 1788863361.78 total 10.02 last 13.37)
 - mkdir 279 (time 1788863371.79 total 20.03 last 14.49)
 - mkdir 416 (time 1788863381.79 total 30.04 last 13.69)
 - mkdir 579 (time 1788863391.84 total 40.09 last 16.22)
 - mkdir 748 (time 1788863401.93 total 50.17 last 16.76)
 - mkdir 917 (time 1788863411.96 total 60.20 last 16.84)
total: 1000 mkdir in 68.41 seconds: 14.62 ops/second
 - mkdir 133 (time 1788863361.71 total 10.05 last 13.23)
 - mkdir 275 (time 1788863371.76 total 20.10 last 14.13)
 - mkdir 412 (time 1788863381.80 total 30.14 last 13.65)
 - mkdir 573 (time 1788863391.85 total 40.19 last 16.01)
 - mkdir 747 (time 1788863401.92 total 50.26 last 17.29)
 - mkdir 916 (time 1788863411.95 total 60.29 last 16.85)
total: 1000 mkdir in 68.81 seconds: 14.53 ops/second
Time for file creation 81 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788863428 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788863429 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788863430 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788863429 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788863430 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788863430 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1788863430 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788863430 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788863431 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788863430 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
Time for file removal 48 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8