== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 21:36:10 (1788485770)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.64 seconds: 15.70 ops/second
 - mkdir 168 (time 1788485793.00 total 10.02 last 16.77)
 - mkdir 325 (time 1788485803.03 total 20.05 last 15.65)
 - mkdir 462 (time 1788485813.04 total 30.06 last 13.69)
 - mkdir 634 (time 1788485823.11 total 40.14 last 17.07)
 - mkdir 781 (time 1788485833.14 total 50.16 last 14.66)
 - mkdir 925 (time 1788485843.19 total 60.21 last 14.33)
total: 1000 mkdir in 65.72 seconds: 15.22 ops/second
 - mkdir 167 (time 1788485793.57 total 10.08 last 16.56)
 - mkdir 325 (time 1788485803.58 total 20.09 last 15.79)
 - mkdir 467 (time 1788485813.63 total 30.14 last 14.13)
 - mkdir 635 (time 1788485823.67 total 40.18 last 16.73)
 - mkdir 787 (time 1788485833.69 total 50.20 last 15.18)
 - mkdir 933 (time 1788485843.72 total 60.23 last 14.55)
total: 1000 mkdir in 65.61 seconds: 15.24 ops/second
 - mkdir 181 (time 1788485794.79 total 10.01 last 18.08)
 - mkdir 333 (time 1788485804.81 total 20.04 last 15.16)
 - mkdir 477 (time 1788485814.88 total 30.11 last 14.30)
 - mkdir 651 (time 1788485824.91 total 40.13 last 17.35)
 - mkdir 799 (time 1788485834.99 total 50.21 last 14.69)
 - mkdir 951 (time 1788485845.00 total 60.22 last 15.19)
total: 1000 mkdir in 64.66 seconds: 15.46 ops/second
 - mkdir 173 (time 1788485794.16 total 10.03 last 17.25)
 - mkdir 323 (time 1788485804.19 total 20.06 last 14.96)
 - mkdir 464 (time 1788485814.23 total 30.10 last 14.04)
 - mkdir 636 (time 1788485824.25 total 40.11 last 17.18)
 - mkdir 786 (time 1788485834.28 total 50.15 last 14.95)
 - mkdir 942 (time 1788485844.35 total 60.22 last 15.50)
total: 1000 mkdir in 65.41 seconds: 15.29 ops/second
 - mkdir 175 (time 1788485794.73 total 10.00 last 17.50)
 - mkdir 324 (time 1788485804.76 total 20.04 last 14.85)
 - mkdir 474 (time 1788485814.77 total 30.05 last 14.99)
 - mkdir 645 (time 1788485824.83 total 40.11 last 16.99)
 - mkdir 791 (time 1788485834.94 total 50.21 last 14.46)
 - mkdir 943 (time 1788485844.99 total 60.27 last 15.11)
total: 1000 mkdir in 65.65 seconds: 15.23 ops/second
 - mkdir 177 (time 1788485794.64 total 10.04 last 17.62)
 - mkdir 325 (time 1788485804.68 total 20.08 last 14.74)
 - mkdir 470 (time 1788485814.77 total 30.17 last 14.37)
 - mkdir 640 (time 1788485824.83 total 40.24 last 16.89)
 - mkdir 791 (time 1788485834.91 total 50.32 last 14.98)
 - mkdir 941 (time 1788485844.97 total 60.38 last 14.90)
total: 1000 mkdir in 65.82 seconds: 15.19 ops/second
 - mkdir 169 (time 1788485794.08 total 10.05 last 16.81)
 - mkdir 321 (time 1788485804.16 total 20.13 last 15.09)
 - mkdir 464 (time 1788485814.20 total 30.17 last 14.24)
 - mkdir 631 (time 1788485824.23 total 40.20 last 16.64)
 - mkdir 783 (time 1788485834.29 total 50.26 last 15.11)
 - mkdir 931 (time 1788485844.32 total 60.29 last 14.76)
total: 1000 mkdir in 66.44 seconds: 15.05 ops/second
 - mkdir 174 (time 1788485794.56 total 10.02 last 17.37)
 - mkdir 322 (time 1788485804.63 total 20.09 last 14.70)
 - mkdir 470 (time 1788485814.69 total 30.14 last 14.71)
 - mkdir 641 (time 1788485824.73 total 40.19 last 17.02)
 - mkdir 790 (time 1788485834.80 total 50.26 last 14.80)
 - mkdir 940 (time 1788485844.84 total 60.30 last 14.93)
total: 1000 mkdir in 65.98 seconds: 15.16 ops/second
 - mkdir 181 (time 1788485794.81 total 10.04 last 18.03)
 - mkdir 327 (time 1788485804.84 total 20.07 last 14.55)
 - mkdir 470 (time 1788485814.85 total 30.08 last 14.29)
 - mkdir 642 (time 1788485824.89 total 40.12 last 17.13)
 - mkdir 789 (time 1788485834.90 total 50.13 last 14.69)
 - mkdir 938 (time 1788485844.90 total 60.13 last 14.90)
total: 1000 mkdir in 66.12 seconds: 15.12 ops/second
 - mkdir 174 (time 1788485794.76 total 10.05 last 17.32)
 - mkdir 327 (time 1788485804.78 total 20.06 last 15.28)
 - mkdir 469 (time 1788485814.80 total 30.08 last 14.17)
 - mkdir 638 (time 1788485824.86 total 40.15 last 16.79)
 - mkdir 786 (time 1788485834.92 total 50.20 last 14.72)
 - mkdir 936 (time 1788485845.02 total 60.30 last 14.85)
total: 1000 mkdir in 66.28 seconds: 15.09 ops/second
Time for file creation 80 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 14 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.67 seconds: 14.90 ops/second
 - mkdir 135 (time 1788486440.37 total 10.02 last 13.48)
 - mkdir 259 (time 1788486450.37 total 20.02 last 12.40)
 - mkdir 431 (time 1788486460.38 total 30.03 last 17.19)
 - mkdir 593 (time 1788486470.43 total 40.08 last 16.12)
 - mkdir 742 (time 1788486480.45 total 50.10 last 14.86)
 - mkdir 879 (time 1788486490.51 total 60.16 last 13.62)
total: 1000 mkdir in 67.93 seconds: 14.72 ops/second
 - mkdir 132 (time 1788486441.27 total 10.00 last 13.19)
 - mkdir 268 (time 1788486451.32 total 20.05 last 13.54)
 - mkdir 440 (time 1788486461.33 total 30.06 last 17.18)
 - mkdir 604 (time 1788486471.37 total 40.10 last 16.34)
 - mkdir 758 (time 1788486481.41 total 50.14 last 15.33)
 - mkdir 886 (time 1788486491.45 total 60.18 last 12.75)
total: 1000 mkdir in 67.44 seconds: 14.83 ops/second
 - mkdir 138 (time 1788486440.79 total 10.11 last 13.65)
 - mkdir 270 (time 1788486450.79 total 20.12 last 13.19)
 - mkdir 438 (time 1788486460.82 total 30.14 last 16.76)
 - mkdir 598 (time 1788486470.87 total 40.19 last 15.92)
 - mkdir 750 (time 1788486480.90 total 50.23 last 15.15)
 - mkdir 879 (time 1788486490.93 total 60.25 last 12.87)
total: 1000 mkdir in 68.07 seconds: 14.69 ops/second
 - mkdir 137 (time 1788486441.24 total 10.00 last 13.70)
 - mkdir 271 (time 1788486451.26 total 20.02 last 13.37)
 - mkdir 440 (time 1788486461.34 total 30.10 last 16.77)
 - mkdir 604 (time 1788486471.39 total 40.16 last 16.31)
 - mkdir 755 (time 1788486481.42 total 50.19 last 15.05)
 - mkdir 887 (time 1788486491.49 total 60.26 last 13.11)
total: 1000 mkdir in 67.82 seconds: 14.74 ops/second
 - mkdir 135 (time 1788486440.83 total 10.04 last 13.44)
 - mkdir 263 (time 1788486450.87 total 20.08 last 12.76)
 - mkdir 434 (time 1788486460.88 total 30.09 last 17.07)
 - mkdir 593 (time 1788486470.89 total 40.10 last 15.89)
 - mkdir 745 (time 1788486480.90 total 50.11 last 15.18)
 - mkdir 880 (time 1788486490.93 total 60.14 last 13.46)
total: 1000 mkdir in 68.34 seconds: 14.63 ops/second
 - mkdir 136 (time 1788486441.56 total 10.01 last 13.59)
 - mkdir 268 (time 1788486451.69 total 20.13 last 13.04)
 - mkdir 437 (time 1788486461.70 total 30.14 last 16.88)
 - mkdir 600 (time 1788486471.78 total 40.22 last 16.17)
 - mkdir 751 (time 1788486481.80 total 50.24 last 15.07)
 - mkdir 883 (time 1788486491.81 total 60.25 last 13.18)
total: 1000 mkdir in 68.30 seconds: 14.64 ops/second
 - mkdir 130 (time 1788486441.17 total 10.00 last 13.00)
 - mkdir 265 (time 1788486451.22 total 20.04 last 13.44)
 - mkdir 433 (time 1788486461.28 total 30.10 last 16.70)
 - mkdir 596 (time 1788486471.30 total 40.13 last 16.26)
 - mkdir 748 (time 1788486481.33 total 50.16 last 15.15)
 - mkdir 877 (time 1788486491.37 total 60.19 last 12.86)
total: 1000 mkdir in 68.84 seconds: 14.53 ops/second
 - mkdir 134 (time 1788486441.39 total 10.07 last 13.31)
 - mkdir 264 (time 1788486451.42 total 20.10 last 12.96)
 - mkdir 434 (time 1788486461.46 total 30.14 last 16.94)
 - mkdir 599 (time 1788486471.52 total 40.19 last 16.40)
 - mkdir 750 (time 1788486481.57 total 50.24 last 15.03)
 - mkdir 880 (time 1788486491.59 total 60.26 last 12.97)
total: 1000 mkdir in 69.02 seconds: 14.49 ops/second
 - mkdir 126 (time 1788486439.32 total 10.01 last 12.59)
 - mkdir 254 (time 1788486449.50 total 20.18 last 12.58)
 - mkdir 415 (time 1788486459.51 total 30.19 last 16.08)
 - mkdir 570 (time 1788486469.52 total 40.20 last 15.48)
 - mkdir 718 (time 1788486479.59 total 50.27 last 14.70)
 - mkdir 855 (time 1788486489.66 total 60.34 last 13.61)
 - mkdir 989 (time 1788486499.69 total 70.37 last 13.36)
total: 1000 mkdir in 71.43 seconds: 14.00 ops/second
 - mkdir 132 (time 1788486441.35 total 10.04 last 13.15)
 - mkdir 261 (time 1788486451.36 total 20.04 last 12.90)
 - mkdir 424 (time 1788486461.43 total 30.11 last 16.18)
 - mkdir 586 (time 1788486471.48 total 40.16 last 16.12)
 - mkdir 739 (time 1788486481.55 total 50.23 last 15.20)
 - mkdir 866 (time 1788486491.56 total 60.24 last 12.69)
total: 1000 mkdir in 70.20 seconds: 14.25 ops/second
Time for file creation 81 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788486512 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788486512 ; total 1 ; last 1)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788486512 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788486513 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788486513 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788486513 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788486512 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1788486513 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788486513 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788486513 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
Time for file removal 48 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8