== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 04:45:03 (1789289103)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.59 seconds: 16.90 ops/second
 - mkdir 111 (time 1789289124.28 total 10.03 last 11.07)
 - mkdir 243 (time 1789289134.34 total 20.10 last 13.11)
 - mkdir 382 (time 1789289144.35 total 30.10 last 13.89)
 - mkdir 496 (time 1789289154.41 total 40.16 last 11.33)
 - mkdir 638 (time 1789289164.49 total 50.24 last 14.09)
 - mkdir 801 (time 1789289174.59 total 60.34 last 16.14)
 - mkdir 972 (time 1789289184.62 total 70.37 last 17.05)
total: 1000 mkdir in 71.68 seconds: 13.95 ops/second
 - mkdir 119 (time 1789289124.25 total 10.08 last 11.80)
 - mkdir 254 (time 1789289134.28 total 20.12 last 13.45)
 - mkdir 393 (time 1789289144.30 total 30.13 last 13.88)
 - mkdir 500 (time 1789289154.30 total 40.14 last 10.69)
 - mkdir 635 (time 1789289164.35 total 50.18 last 13.44)
 - mkdir 789 (time 1789289174.38 total 60.22 last 15.35)
 - mkdir 962 (time 1789289184.43 total 70.26 last 17.22)
total: 1000 mkdir in 72.05 seconds: 13.88 ops/second
 - mkdir 123 (time 1789289127.91 total 10.07 last 12.22)
 - mkdir 267 (time 1789289137.92 total 20.09 last 14.38)
 - mkdir 400 (time 1789289147.95 total 30.12 last 13.26)
 - mkdir 520 (time 1789289158.00 total 40.16 last 11.95)
 - mkdir 671 (time 1789289168.01 total 50.17 last 15.09)
 - mkdir 829 (time 1789289178.01 total 60.17 last 15.80)
total: 1000 mkdir in 69.27 seconds: 14.44 ops/second
 - mkdir 116 (time 1789289128.04 total 10.05 last 11.54)
 - mkdir 261 (time 1789289138.07 total 20.09 last 14.45)
 - mkdir 394 (time 1789289148.11 total 30.13 last 13.24)
 - mkdir 505 (time 1789289158.16 total 40.18 last 11.05)
 - mkdir 651 (time 1789289168.22 total 50.23 last 14.52)
 - mkdir 810 (time 1789289178.24 total 60.26 last 15.86)
 - mkdir 984 (time 1789289188.35 total 70.36 last 17.22)
total: 1000 mkdir in 72.88 seconds: 13.72 ops/second
 - mkdir 126 (time 1789289129.00 total 10.00 last 12.59)
 - mkdir 270 (time 1789289139.07 total 20.07 last 14.31)
 - mkdir 395 (time 1789289149.17 total 30.17 last 12.38)
 - mkdir 512 (time 1789289159.22 total 40.22 last 11.64)
 - mkdir 669 (time 1789289169.24 total 50.24 last 15.67)
 - mkdir 827 (time 1789289179.26 total 60.26 last 15.76)
 - mkdir 987 (time 1789289189.31 total 70.31 last 15.92)
total: 1000 mkdir in 72.35 seconds: 13.82 ops/second
 - mkdir 114 (time 1789289127.87 total 10.04 last 11.35)
 - mkdir 254 (time 1789289137.90 total 20.07 last 13.97)
 - mkdir 381 (time 1789289147.98 total 30.15 last 12.59)
 - mkdir 497 (time 1789289157.99 total 40.16 last 11.59)
 - mkdir 641 (time 1789289168.02 total 50.19 last 14.36)
 - mkdir 799 (time 1789289178.03 total 60.20 last 15.78)
 - mkdir 977 (time 1789289188.06 total 70.23 last 17.76)
total: 1000 mkdir in 73.62 seconds: 13.58 ops/second
 - mkdir 123 (time 1789289129.17 total 10.00 last 12.29)
 - mkdir 265 (time 1789289139.20 total 20.04 last 14.16)
 - mkdir 385 (time 1789289149.24 total 30.08 last 11.95)
 - mkdir 500 (time 1789289159.27 total 40.11 last 11.47)
 - mkdir 650 (time 1789289169.28 total 50.12 last 14.99)
 - mkdir 804 (time 1789289179.29 total 60.12 last 15.39)
 - mkdir 972 (time 1789289189.30 total 70.14 last 16.78)
total: 1000 mkdir in 73.83 seconds: 13.54 ops/second
 - mkdir 124 (time 1789289129.00 total 10.10 last 12.28)
 - mkdir 268 (time 1789289139.00 total 20.11 last 14.39)
 - mkdir 388 (time 1789289149.01 total 30.11 last 12.00)
 - mkdir 499 (time 1789289159.04 total 40.14 last 11.07)
 - mkdir 646 (time 1789289169.05 total 50.15 last 14.68)
 - mkdir 806 (time 1789289179.08 total 60.18 last 15.95)
 - mkdir 965 (time 1789289189.17 total 70.27 last 15.76)
total: 1000 mkdir in 74.55 seconds: 13.41 ops/second
 - mkdir 117 (time 1789289128.55 total 10.06 last 11.63)
 - mkdir 252 (time 1789289138.55 total 20.07 last 13.49)
 - mkdir 376 (time 1789289148.56 total 30.08 last 12.39)
 - mkdir 482 (time 1789289158.57 total 40.09 last 10.58)
 - mkdir 629 (time 1789289168.59 total 50.11 last 14.68)
 - mkdir 782 (time 1789289178.60 total 60.12 last 15.28)
 - mkdir 954 (time 1789289188.60 total 70.12 last 17.20)
total: 1000 mkdir in 75.42 seconds: 13.26 ops/second
 - mkdir 108 (time 1789289127.13 total 10.05 last 10.75)
 - mkdir 246 (time 1789289137.21 total 20.12 last 13.69)
 - mkdir 365 (time 1789289147.22 total 30.14 last 11.88)
 - mkdir 470 (time 1789289157.23 total 40.14 last 10.50)
 - mkdir 608 (time 1789289167.26 total 50.17 last 13.76)
 - mkdir 761 (time 1789289177.36 total 60.28 last 15.14)
 - mkdir 935 (time 1789289187.43 total 70.35 last 17.28)
total: 1000 mkdir in 77.30 seconds: 12.94 ops/second
Time for file creation 90 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
Time for file removal 16 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.40 seconds: 25.25 ops/second
 - mkdir 149 (time 1789289755.89 total 10.01 last 14.89)
 - mkdir 294 (time 1789289765.93 total 20.04 last 14.44)
 - mkdir 452 (time 1789289775.95 total 30.07 last 15.76)
 - mkdir 623 (time 1789289785.95 total 40.07 last 17.10)
 - mkdir 806 (time 1789289795.98 total 50.10 last 18.25)
 - mkdir 985 (time 1789289805.99 total 60.11 last 17.88)
total: 1000 mkdir in 61.05 seconds: 16.38 ops/second
 - mkdir 155 (time 1789289757.71 total 10.06 last 15.41)
 - mkdir 303 (time 1789289767.73 total 20.08 last 14.77)
 - mkdir 454 (time 1789289777.82 total 30.17 last 14.97)
 - mkdir 632 (time 1789289787.88 total 40.22 last 17.70)
 - mkdir 825 (time 1789289797.89 total 50.24 last 19.27)
 - mkdir 993 (time 1789289807.92 total 60.27 last 16.76)
total: 1000 mkdir in 60.75 seconds: 16.46 ops/second
 - mkdir 147 (time 1789289756.84 total 10.02 last 14.66)
 - mkdir 285 (time 1789289766.86 total 20.04 last 13.77)
 - mkdir 434 (time 1789289776.87 total 30.06 last 14.87)
 - mkdir 606 (time 1789289786.92 total 40.11 last 17.13)
 - mkdir 796 (time 1789289796.92 total 50.11 last 18.99)
 - mkdir 972 (time 1789289806.94 total 60.13 last 17.57)
total: 1000 mkdir in 62.56 seconds: 15.98 ops/second
 - mkdir 159 (time 1789289758.53 total 10.03 last 15.85)
 - mkdir 307 (time 1789289768.62 total 20.12 last 14.68)
 - mkdir 472 (time 1789289778.67 total 30.17 last 16.42)
 - mkdir 642 (time 1789289788.69 total 40.19 last 16.96)
 - mkdir 835 (time 1789289798.73 total 50.23 last 19.23)
 - mkdir 993 (time 1789289808.78 total 60.28 last 15.72)
total: 1000 mkdir in 60.96 seconds: 16.40 ops/second
 - mkdir 151 (time 1789289758.21 total 10.05 last 15.02)
 - mkdir 296 (time 1789289768.22 total 20.07 last 14.47)
 - mkdir 460 (time 1789289778.31 total 30.16 last 16.27)
 - mkdir 632 (time 1789289788.37 total 40.21 last 17.10)
 - mkdir 824 (time 1789289798.39 total 50.24 last 19.15)
 - mkdir 990 (time 1789289808.47 total 60.32 last 16.47)
total: 1000 mkdir in 61.50 seconds: 16.26 ops/second
 - mkdir 153 (time 1789289758.33 total 10.05 last 15.23)
 - mkdir 299 (time 1789289768.38 total 20.10 last 14.52)
 - mkdir 465 (time 1789289778.47 total 30.19 last 16.45)
 - mkdir 633 (time 1789289788.53 total 40.25 last 16.70)
 - mkdir 829 (time 1789289798.54 total 50.25 last 19.59)
 - mkdir 989 (time 1789289808.59 total 60.31 last 15.91)
total: 1000 mkdir in 61.50 seconds: 16.26 ops/second
 - mkdir 159 (time 1789289759.04 total 10.01 last 15.88)
 - mkdir 313 (time 1789289769.06 total 20.04 last 15.37)
 - mkdir 477 (time 1789289779.11 total 30.08 last 16.33)
 - mkdir 641 (time 1789289789.11 total 40.09 last 16.39)
 - mkdir 837 (time 1789289799.17 total 50.14 last 19.50)
 - mkdir 989 (time 1789289809.24 total 60.21 last 15.09)
total: 1000 mkdir in 61.44 seconds: 16.28 ops/second
 - mkdir 147 (time 1789289758.70 total 10.08 last 14.59)
 - mkdir 294 (time 1789289768.75 total 20.12 last 14.63)
 - mkdir 456 (time 1789289778.78 total 30.15 last 16.15)
 - mkdir 623 (time 1789289788.83 total 40.20 last 16.62)
 - mkdir 821 (time 1789289798.86 total 50.24 last 19.73)
 - mkdir 981 (time 1789289808.90 total 60.28 last 15.93)
total: 1000 mkdir in 62.31 seconds: 16.05 ops/second
 - mkdir 148 (time 1789289758.76 total 10.02 last 14.77)
 - mkdir 298 (time 1789289768.81 total 20.07 last 14.93)
 - mkdir 463 (time 1789289778.84 total 30.11 last 16.44)
 - mkdir 629 (time 1789289788.89 total 40.15 last 16.53)
 - mkdir 825 (time 1789289798.94 total 50.21 last 19.49)
 - mkdir 980 (time 1789289808.98 total 60.24 last 15.44)
total: 1000 mkdir in 62.44 seconds: 16.02 ops/second
 - mkdir 151 (time 1789289758.92 total 10.03 last 15.06)
 - mkdir 299 (time 1789289768.96 total 20.06 last 14.75)
 - mkdir 458 (time 1789289778.96 total 30.07 last 15.89)
 - mkdir 628 (time 1789289788.97 total 40.07 last 16.99)
 - mkdir 822 (time 1789289798.98 total 50.08 last 19.38)
 - mkdir 979 (time 1789289809.07 total 60.17 last 15.56)
total: 1000 mkdir in 62.30 seconds: 16.05 ops/second
Time for file creation 75 sec for 1 RPC sent at a time
 - unlinked 0 (time 1789289818 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1789289819 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1789289819 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1789289820 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1789289820 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1789289820 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1789289819 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1789289820 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1789289820 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1789289820 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
Time for file removal 43 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8