== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 08:59:23 (1789477163)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.59 seconds: 17.09 ops/second
 - mkdir 125 (time 1789477185.74 total 10.02 last 12.48)
 - mkdir 261 (time 1789477195.78 total 20.05 last 13.55)
 - mkdir 424 (time 1789477205.81 total 30.08 last 16.25)
 - mkdir 579 (time 1789477215.83 total 40.11 last 15.47)
 - mkdir 706 (time 1789477225.84 total 50.12 last 12.68)
 - mkdir 842 (time 1789477235.90 total 60.17 last 13.53)
total: 1000 mkdir in 66.99 seconds: 14.93 ops/second
 - mkdir 151 (time 1789477187.86 total 10.01 last 15.09)
 - mkdir 295 (time 1789477197.89 total 20.04 last 14.35)
 - mkdir 464 (time 1789477207.93 total 30.07 last 16.84)
 - mkdir 610 (time 1789477217.99 total 40.13 last 14.51)
 - mkdir 729 (time 1789477228.05 total 50.20 last 11.82)
 - mkdir 899 (time 1789477238.08 total 60.22 last 16.96)
total: 1000 mkdir in 65.71 seconds: 15.22 ops/second
 - mkdir 134 (time 1789477186.15 total 10.00 last 13.40)
 - mkdir 280 (time 1789477196.17 total 20.03 last 14.57)
 - mkdir 443 (time 1789477206.19 total 30.04 last 16.28)
 - mkdir 594 (time 1789477216.25 total 40.10 last 15.01)
 - mkdir 717 (time 1789477226.31 total 50.16 last 12.23)
 - mkdir 852 (time 1789477236.31 total 60.16 last 13.49)
total: 1000 mkdir in 67.70 seconds: 14.77 ops/second
 - mkdir 148 (time 1789477188.29 total 10.01 last 14.79)
 - mkdir 300 (time 1789477198.35 total 20.07 last 15.10)
 - mkdir 472 (time 1789477208.37 total 30.09 last 17.16)
 - mkdir 611 (time 1789477218.44 total 40.16 last 13.81)
 - mkdir 728 (time 1789477228.48 total 50.20 last 11.65)
 - mkdir 902 (time 1789477238.52 total 60.24 last 17.33)
total: 1000 mkdir in 65.97 seconds: 15.16 ops/second
 - mkdir 125 (time 1789477185.25 total 10.12 last 12.36)
 - mkdir 254 (time 1789477195.25 total 20.12 last 12.90)
 - mkdir 413 (time 1789477205.28 total 30.14 last 15.86)
 - mkdir 572 (time 1789477215.31 total 40.18 last 15.84)
 - mkdir 692 (time 1789477225.36 total 50.22 last 11.95)
 - mkdir 826 (time 1789477235.40 total 60.27 last 13.34)
total: 1000 mkdir in 69.57 seconds: 14.37 ops/second
 - mkdir 157 (time 1789477188.60 total 10.11 last 15.53)
 - mkdir 307 (time 1789477198.62 total 20.13 last 14.96)
 - mkdir 475 (time 1789477208.66 total 30.17 last 16.74)
 - mkdir 613 (time 1789477218.66 total 40.17 last 13.80)
 - mkdir 729 (time 1789477228.67 total 50.17 last 11.60)
 - mkdir 904 (time 1789477238.67 total 60.18 last 17.49)
total: 1000 mkdir in 66.24 seconds: 15.10 ops/second
 - mkdir 141 (time 1789477187.24 total 10.08 last 13.99)
 - mkdir 284 (time 1789477197.28 total 20.11 last 14.25)
 - mkdir 444 (time 1789477207.29 total 30.12 last 15.98)
 - mkdir 590 (time 1789477217.29 total 40.12 last 14.60)
 - mkdir 709 (time 1789477227.30 total 50.14 last 11.88)
 - mkdir 865 (time 1789477237.31 total 60.15 last 15.59)
total: 1000 mkdir in 67.73 seconds: 14.76 ops/second
 - mkdir 143 (time 1789477187.08 total 10.02 last 14.27)
 - mkdir 287 (time 1789477197.11 total 20.06 last 14.35)
 - mkdir 446 (time 1789477207.16 total 30.10 last 15.83)
 - mkdir 593 (time 1789477217.17 total 40.11 last 14.68)
 - mkdir 712 (time 1789477227.23 total 50.17 last 11.83)
 - mkdir 863 (time 1789477237.23 total 60.18 last 15.10)
total: 1000 mkdir in 68.41 seconds: 14.62 ops/second
 - mkdir 141 (time 1789477187.56 total 10.03 last 14.06)
 - mkdir 279 (time 1789477197.60 total 20.07 last 13.74)
 - mkdir 445 (time 1789477207.63 total 30.10 last 16.55)
 - mkdir 588 (time 1789477217.66 total 40.13 last 14.26)
 - mkdir 701 (time 1789477227.70 total 50.17 last 11.25)
 - mkdir 863 (time 1789477237.73 total 60.20 last 16.15)
total: 1000 mkdir in 68.99 seconds: 14.49 ops/second
 - mkdir 137 (time 1789477187.50 total 10.07 last 13.60)
 - mkdir 272 (time 1789477197.57 total 20.15 last 13.40)
 - mkdir 439 (time 1789477207.62 total 30.20 last 16.62)
 - mkdir 589 (time 1789477217.65 total 40.23 last 14.95)
 - mkdir 705 (time 1789477227.66 total 50.23 last 11.60)
 - mkdir 862 (time 1789477237.71 total 60.28 last 15.62)
total: 1000 mkdir in 69.86 seconds: 14.31 ops/second
Time for file creation 82 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 15 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.52 seconds: 19.36 ops/second
 - mkdir 161 (time 1789477793.89 total 10.06 last 16.00)
 - mkdir 305 (time 1789477803.94 total 20.11 last 14.33)
 - mkdir 465 (time 1789477813.96 total 30.13 last 15.96)
 - mkdir 619 (time 1789477823.98 total 40.15 last 15.37)
 - mkdir 780 (time 1789477834.01 total 50.17 last 16.06)
 - mkdir 952 (time 1789477844.08 total 60.25 last 17.07)
total: 1000 mkdir in 63.14 seconds: 15.84 ops/second
 - mkdir 150 (time 1789477792.37 total 10.03 last 14.96)
 - mkdir 292 (time 1789477802.38 total 20.03 last 14.19)
 - mkdir 433 (time 1789477812.41 total 30.06 last 14.06)
 - mkdir 586 (time 1789477822.43 total 40.08 last 15.27)
 - mkdir 745 (time 1789477832.45 total 50.11 last 15.86)
 - mkdir 916 (time 1789477842.49 total 60.14 last 17.03)
total: 1000 mkdir in 65.73 seconds: 15.21 ops/second
 - mkdir 155 (time 1789477793.82 total 10.04 last 15.44)
 - mkdir 297 (time 1789477803.86 total 20.08 last 14.15)
 - mkdir 453 (time 1789477813.86 total 30.08 last 15.60)
 - mkdir 610 (time 1789477823.90 total 40.12 last 15.64)
 - mkdir 771 (time 1789477833.99 total 50.20 last 15.96)
 - mkdir 944 (time 1789477844.09 total 60.30 last 17.12)
total: 1000 mkdir in 64.62 seconds: 15.48 ops/second
 - mkdir 153 (time 1789477794.48 total 10.00 last 15.30)
 - mkdir 293 (time 1789477804.48 total 20.01 last 13.99)
 - mkdir 446 (time 1789477814.52 total 30.04 last 15.24)
 - mkdir 598 (time 1789477824.53 total 40.05 last 15.18)
 - mkdir 747 (time 1789477834.57 total 50.09 last 14.84)
 - mkdir 922 (time 1789477844.57 total 60.09 last 17.50)
total: 1000 mkdir in 67.19 seconds: 14.88 ops/second
 - mkdir 148 (time 1789477794.26 total 10.05 last 14.72)
 - mkdir 288 (time 1789477804.30 total 20.10 last 13.94)
 - mkdir 442 (time 1789477814.36 total 30.15 last 15.32)
 - mkdir 596 (time 1789477824.36 total 40.16 last 15.39)
 - mkdir 748 (time 1789477834.43 total 50.22 last 15.11)
 - mkdir 924 (time 1789477844.47 total 60.27 last 17.52)
total: 1000 mkdir in 67.84 seconds: 14.74 ops/second
 - mkdir 127 (time 1789477791.20 total 10.01 last 12.69)
 - mkdir 265 (time 1789477801.23 total 20.04 last 13.76)
 - mkdir 406 (time 1789477811.26 total 30.07 last 14.06)
 - mkdir 552 (time 1789477821.29 total 40.10 last 14.56)
 - mkdir 705 (time 1789477831.33 total 50.13 last 15.25)
 - mkdir 856 (time 1789477841.37 total 60.18 last 15.03)
 - mkdir 986 (time 1789477851.38 total 70.18 last 12.99)
total: 1000 mkdir in 71.69 seconds: 13.95 ops/second
 - mkdir 150 (time 1789477794.27 total 10.02 last 14.96)
 - mkdir 286 (time 1789477804.28 total 20.04 last 13.58)
 - mkdir 433 (time 1789477814.32 total 30.07 last 14.65)
 - mkdir 583 (time 1789477824.34 total 40.10 last 14.96)
 - mkdir 733 (time 1789477834.42 total 50.18 last 14.89)
 - mkdir 905 (time 1789477844.43 total 60.18 last 17.18)
total: 1000 mkdir in 69.55 seconds: 14.38 ops/second
 - mkdir 155 (time 1789477794.58 total 10.08 last 15.37)
 - mkdir 288 (time 1789477804.62 total 20.12 last 13.26)
 - mkdir 436 (time 1789477814.64 total 30.14 last 14.77)
 - mkdir 586 (time 1789477824.64 total 40.14 last 14.99)
 - mkdir 731 (time 1789477834.70 total 50.20 last 14.42)
 - mkdir 907 (time 1789477844.72 total 60.22 last 17.55)
total: 1000 mkdir in 69.66 seconds: 14.36 ops/second
 - mkdir 157 (time 1789477794.99 total 10.03 last 15.66)
 - mkdir 292 (time 1789477805.08 total 20.12 last 13.38)
 - mkdir 443 (time 1789477815.11 total 30.15 last 15.05)
 - mkdir 596 (time 1789477825.15 total 40.18 last 15.25)
 - mkdir 744 (time 1789477835.17 total 50.20 last 14.77)
 - mkdir 918 (time 1789477845.18 total 60.22 last 17.38)
total: 1000 mkdir in 69.28 seconds: 14.43 ops/second
 - mkdir 142 (time 1789477794.96 total 10.03 last 14.15)
 - mkdir 276 (time 1789477805.02 total 20.09 last 13.33)
 - mkdir 425 (time 1789477815.02 total 30.10 last 14.89)
 - mkdir 573 (time 1789477825.06 total 40.14 last 14.74)
 - mkdir 710 (time 1789477835.09 total 50.16 last 13.66)
 - mkdir 879 (time 1789477845.18 total 60.25 last 16.75)
 - mkdir 974 (time 1789477855.18 total 70.26 last 9.50)
total: 1000 mkdir in 72.24 seconds: 13.84 ops/second
Time for file creation 86 sec for 1 RPC sent at a time
 - unlinked 0 (time 1789477865 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1789477865 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1789477868 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789477867 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1789477867 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1789477867 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1789477867 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1789477867 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1789477868 ; total 1 ; last 1)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1789477868 ; total 1 ; last 1)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
Time for file removal 46 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8