== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 16:49:57 (1788641397)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.51 seconds: 19.54 ops/second
 - mkdir 159 (time 1788641415.61 total 10.01 last 15.89)
 - mkdir 331 (time 1788641425.69 total 20.09 last 17.06)
 - mkdir 493 (time 1788641435.69 total 30.09 last 16.19)
 - mkdir 648 (time 1788641445.74 total 40.14 last 15.42)
 - mkdir 838 (time 1788641455.81 total 50.20 last 18.88)
total: 1000 mkdir in 60.07 seconds: 16.65 ops/second
 - mkdir 183 (time 1788641418.66 total 10.03 last 18.24)
 - mkdir 361 (time 1788641428.72 total 20.09 last 17.69)
 - mkdir 527 (time 1788641438.78 total 30.15 last 16.51)
 - mkdir 700 (time 1788641448.79 total 40.16 last 17.27)
 - mkdir 887 (time 1788641458.80 total 50.17 last 18.69)
total: 1000 mkdir in 57.30 seconds: 17.45 ops/second
 - mkdir 177 (time 1788641418.67 total 10.06 last 17.59)
 - mkdir 348 (time 1788641428.69 total 20.08 last 17.07)
 - mkdir 510 (time 1788641438.74 total 30.13 last 16.11)
 - mkdir 683 (time 1788641448.75 total 40.14 last 17.29)
 - mkdir 875 (time 1788641458.79 total 50.18 last 19.12)
total: 1000 mkdir in 58.36 seconds: 17.14 ops/second
 - mkdir 174 (time 1788641418.01 total 10.01 last 17.38)
 - mkdir 342 (time 1788641428.02 total 20.03 last 16.77)
 - mkdir 507 (time 1788641438.12 total 30.12 last 16.35)
 - mkdir 674 (time 1788641448.12 total 40.12 last 16.70)
 - mkdir 865 (time 1788641458.14 total 50.14 last 19.06)
total: 1000 mkdir in 59.13 seconds: 16.91 ops/second
 - mkdir 186 (time 1788641419.03 total 10.03 last 18.55)
 - mkdir 355 (time 1788641429.05 total 20.04 last 16.87)
 - mkdir 515 (time 1788641439.10 total 30.09 last 15.92)
 - mkdir 687 (time 1788641449.10 total 40.10 last 17.20)
 - mkdir 878 (time 1788641459.12 total 50.11 last 19.07)
total: 1000 mkdir in 58.50 seconds: 17.09 ops/second
 - mkdir 182 (time 1788641418.95 total 10.01 last 18.17)
 - mkdir 349 (time 1788641428.96 total 20.02 last 16.69)
 - mkdir 511 (time 1788641438.98 total 30.04 last 16.17)
 - mkdir 683 (time 1788641449.03 total 40.10 last 17.11)
 - mkdir 875 (time 1788641459.06 total 50.12 last 19.15)
total: 1000 mkdir in 59.32 seconds: 16.86 ops/second
 - mkdir 177 (time 1788641418.64 total 10.01 last 17.67)
 - mkdir 351 (time 1788641428.65 total 20.02 last 17.39)
 - mkdir 510 (time 1788641438.67 total 30.04 last 15.87)
 - mkdir 676 (time 1788641448.73 total 40.10 last 16.50)
 - mkdir 862 (time 1788641458.74 total 50.12 last 18.58)
total: 1000 mkdir in 60.09 seconds: 16.64 ops/second
 - mkdir 176 (time 1788641418.87 total 10.04 last 17.54)
 - mkdir 341 (time 1788641428.89 total 20.05 last 16.47)
 - mkdir 505 (time 1788641438.97 total 30.13 last 16.28)
 - mkdir 671 (time 1788641449.00 total 40.16 last 16.54)
 - mkdir 859 (time 1788641459.03 total 50.19 last 18.74)
 - mkdir 991 (time 1788641469.09 total 60.25 last 13.13)
total: 1000 mkdir in 61.22 seconds: 16.33 ops/second
 - mkdir 173 (time 1788641419.11 total 10.01 last 17.27)
 - mkdir 335 (time 1788641429.12 total 20.03 last 16.18)
 - mkdir 488 (time 1788641439.17 total 30.08 last 15.22)
 - mkdir 666 (time 1788641449.20 total 40.11 last 17.75)
 - mkdir 859 (time 1788641459.27 total 50.18 last 19.16)
 - mkdir 991 (time 1788641469.37 total 60.28 last 13.07)
total: 1000 mkdir in 61.24 seconds: 16.33 ops/second
 - mkdir 176 (time 1788641418.88 total 10.02 last 17.57)
 - mkdir 337 (time 1788641428.88 total 20.02 last 16.09)
 - mkdir 492 (time 1788641438.91 total 30.05 last 15.45)
 - mkdir 659 (time 1788641448.93 total 40.07 last 16.67)
 - mkdir 851 (time 1788641458.95 total 50.09 last 19.16)
 - mkdir 988 (time 1788641469.05 total 60.18 last 13.58)
total: 1000 mkdir in 61.54 seconds: 16.25 ops/second
Time for file creation 72 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 17 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.51 seconds: 19.50 ops/second
 - mkdir 196 (time 1788642001.05 total 10.03 last 19.55)
 - mkdir 359 (time 1788642011.07 total 20.05 last 16.27)
 - mkdir 520 (time 1788642021.07 total 30.05 last 16.09)
 - mkdir 678 (time 1788642031.12 total 40.10 last 15.73)
 - mkdir 824 (time 1788642041.17 total 50.15 last 14.52)
 - mkdir 985 (time 1788642051.19 total 60.17 last 16.07)
total: 1000 mkdir in 61.41 seconds: 16.28 ops/second
 - mkdir 191 (time 1788642003.05 total 10.04 last 19.03)
 - mkdir 360 (time 1788642013.11 total 20.10 last 16.80)
 - mkdir 526 (time 1788642023.13 total 30.12 last 16.56)
 - mkdir 694 (time 1788642033.15 total 40.14 last 16.77)
 - mkdir 841 (time 1788642043.16 total 50.14 last 14.70)
 - mkdir 990 (time 1788642053.26 total 60.25 last 14.75)
total: 1000 mkdir in 61.50 seconds: 16.26 ops/second
 - mkdir 184 (time 1788642003.20 total 10.03 last 18.35)
 - mkdir 351 (time 1788642013.26 total 20.09 last 16.59)
 - mkdir 518 (time 1788642023.31 total 30.15 last 16.61)
 - mkdir 686 (time 1788642033.35 total 40.18 last 16.75)
 - mkdir 833 (time 1788642043.35 total 50.18 last 14.69)
 - mkdir 979 (time 1788642053.39 total 60.23 last 14.54)
total: 1000 mkdir in 63.13 seconds: 15.84 ops/second
 - mkdir 182 (time 1788642002.93 total 10.01 last 18.18)
 - mkdir 352 (time 1788642012.99 total 20.07 last 16.90)
 - mkdir 515 (time 1788642023.03 total 30.11 last 16.24)
 - mkdir 680 (time 1788642033.04 total 40.11 last 16.49)
 - mkdir 824 (time 1788642043.07 total 50.15 last 14.35)
 - mkdir 970 (time 1788642053.19 total 60.26 last 14.44)
total: 1000 mkdir in 64.49 seconds: 15.51 ops/second
 - mkdir 186 (time 1788642002.67 total 10.02 last 18.57)
 - mkdir 347 (time 1788642012.69 total 20.03 last 16.08)
 - mkdir 508 (time 1788642022.70 total 30.04 last 16.08)
 - mkdir 672 (time 1788642032.73 total 40.07 last 16.34)
 - mkdir 813 (time 1788642042.82 total 50.17 last 13.97)
 - mkdir 964 (time 1788642052.93 total 60.28 last 14.93)
total: 1000 mkdir in 65.30 seconds: 15.31 ops/second
 - mkdir 181 (time 1788642002.28 total 10.02 last 18.06)
 - mkdir 348 (time 1788642012.37 total 20.11 last 16.55)
 - mkdir 506 (time 1788642022.37 total 30.12 last 15.79)
 - mkdir 666 (time 1788642032.39 total 40.14 last 15.97)
 - mkdir 809 (time 1788642042.43 total 50.18 last 14.24)
 - mkdir 960 (time 1788642052.50 total 60.24 last 15.00)
total: 1000 mkdir in 65.71 seconds: 15.22 ops/second
 - mkdir 191 (time 1788642003.20 total 10.01 last 19.09)
 - mkdir 352 (time 1788642013.27 total 20.08 last 15.98)
 - mkdir 513 (time 1788642023.32 total 30.13 last 16.03)
 - mkdir 681 (time 1788642033.34 total 40.15 last 16.77)
 - mkdir 823 (time 1788642043.35 total 50.16 last 14.18)
 - mkdir 967 (time 1788642053.43 total 60.24 last 14.29)
total: 1000 mkdir in 64.95 seconds: 15.40 ops/second
 - mkdir 181 (time 1788642003.05 total 10.05 last 18.02)
 - mkdir 350 (time 1788642013.05 total 20.05 last 16.90)
 - mkdir 508 (time 1788642023.13 total 30.13 last 15.67)
 - mkdir 676 (time 1788642033.17 total 40.17 last 16.73)
 - mkdir 813 (time 1788642043.18 total 50.18 last 13.69)
 - mkdir 957 (time 1788642053.28 total 60.28 last 14.26)
total: 1000 mkdir in 66.17 seconds: 15.11 ops/second
 - mkdir 185 (time 1788642003.07 total 10.05 last 18.41)
 - mkdir 349 (time 1788642013.14 total 20.12 last 16.28)
 - mkdir 507 (time 1788642023.25 total 30.23 last 15.63)
 - mkdir 667 (time 1788642033.26 total 40.24 last 15.99)
 - mkdir 806 (time 1788642043.26 total 50.24 last 13.89)
 - mkdir 946 (time 1788642053.28 total 60.26 last 13.97)
total: 1000 mkdir in 66.67 seconds: 15.00 ops/second
 - mkdir 183 (time 1788642002.60 total 10.03 last 18.24)
 - mkdir 343 (time 1788642012.63 total 20.06 last 15.96)
 - mkdir 500 (time 1788642022.66 total 30.09 last 15.65)
 - mkdir 655 (time 1788642032.69 total 40.12 last 15.45)
 - mkdir 783 (time 1788642042.73 total 50.16 last 12.75)
 - mkdir 927 (time 1788642052.74 total 60.17 last 14.39)
total: 1000 mkdir in 68.70 seconds: 14.56 ops/second
Time for file creation 77 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788642067 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788642067 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788642068 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788642069 ; total 1 ; last 1)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788642069 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788642069 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788642069 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788642069 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788642069 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788642069 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
Time for file removal 38 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8