== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 14:45:37 (1790793937)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.53 seconds: 18.87 ops/second
 - mkdir 130 (time 1790793959.47 total 10.01 last 12.98)
 - mkdir 273 (time 1790793969.58 total 20.12 last 14.15)
 - mkdir 408 (time 1790793979.67 total 30.21 last 13.37)
 - mkdir 560 (time 1790793989.72 total 40.26 last 15.13)
 - mkdir 727 (time 1790793999.72 total 50.26 last 16.69)
 - mkdir 892 (time 1790794009.77 total 60.31 last 16.42)
total: 1000 mkdir in 66.50 seconds: 15.04 ops/second
 - mkdir 134 (time 1790793960.20 total 10.06 last 13.32)
 - mkdir 280 (time 1790793970.21 total 20.07 last 14.59)
 - mkdir 412 (time 1790793980.22 total 30.08 last 13.19)
 - mkdir 565 (time 1790793990.24 total 40.10 last 15.27)
 - mkdir 731 (time 1790794000.27 total 50.12 last 16.56)
 - mkdir 898 (time 1790794010.38 total 60.24 last 16.51)
total: 1000 mkdir in 66.09 seconds: 15.13 ops/second
 - mkdir 131 (time 1790793960.14 total 10.02 last 13.08)
 - mkdir 278 (time 1790793970.21 total 20.08 last 14.60)
 - mkdir 403 (time 1790793980.24 total 30.12 last 12.46)
 - mkdir 560 (time 1790793990.30 total 40.17 last 15.61)
 - mkdir 731 (time 1790794000.34 total 50.22 last 17.02)
 - mkdir 896 (time 1790794010.43 total 60.30 last 16.36)
total: 1000 mkdir in 66.51 seconds: 15.04 ops/second
 - mkdir 142 (time 1790793961.93 total 10.16 last 13.98)
 - mkdir 292 (time 1790793971.96 total 20.18 last 14.96)
 - mkdir 423 (time 1790793981.98 total 30.21 last 13.06)
 - mkdir 586 (time 1790793992.06 total 40.28 last 16.18)
 - mkdir 753 (time 1790794002.08 total 50.31 last 16.66)
 - mkdir 909 (time 1790794012.12 total 60.35 last 15.54)
total: 1000 mkdir in 65.94 seconds: 15.17 ops/second
 - mkdir 137 (time 1790793961.40 total 10.04 last 13.64)
 - mkdir 275 (time 1790793971.43 total 20.07 last 13.76)
 - mkdir 411 (time 1790793981.52 total 30.15 last 13.49)
 - mkdir 576 (time 1790793991.53 total 40.17 last 16.47)
 - mkdir 738 (time 1790794001.57 total 50.21 last 16.14)
 - mkdir 898 (time 1790794011.62 total 60.26 last 15.91)
total: 1000 mkdir in 66.55 seconds: 15.03 ops/second
 - mkdir 133 (time 1790793961.76 total 10.03 last 13.26)
 - mkdir 274 (time 1790793971.79 total 20.06 last 14.07)
 - mkdir 400 (time 1790793981.79 total 30.06 last 12.59)
 - mkdir 566 (time 1790793991.82 total 40.09 last 16.56)
 - mkdir 730 (time 1790794001.84 total 50.11 last 16.37)
 - mkdir 893 (time 1790794011.91 total 60.18 last 16.17)
total: 1000 mkdir in 67.80 seconds: 14.75 ops/second
 - mkdir 140 (time 1790793962.31 total 10.00 last 13.99)
 - mkdir 283 (time 1790793972.32 total 20.02 last 14.28)
 - mkdir 417 (time 1790793982.35 total 30.05 last 13.37)
 - mkdir 580 (time 1790793992.37 total 40.07 last 16.27)
 - mkdir 743 (time 1790794002.40 total 50.10 last 16.25)
 - mkdir 898 (time 1790794012.42 total 60.12 last 15.47)
total: 1000 mkdir in 67.68 seconds: 14.78 ops/second
 - mkdir 135 (time 1790793961.45 total 10.06 last 13.42)
 - mkdir 274 (time 1790793971.51 total 20.12 last 13.82)
 - mkdir 402 (time 1790793981.54 total 30.16 last 12.75)
 - mkdir 569 (time 1790793991.63 total 40.25 last 16.55)
 - mkdir 731 (time 1790794001.68 total 50.29 last 16.13)
 - mkdir 891 (time 1790794011.74 total 60.36 last 15.90)
total: 1000 mkdir in 68.72 seconds: 14.55 ops/second
 - mkdir 131 (time 1790793961.36 total 10.01 last 13.09)
 - mkdir 262 (time 1790793971.39 total 20.04 last 13.06)
 - mkdir 390 (time 1790793981.45 total 30.10 last 12.73)
 - mkdir 552 (time 1790793991.48 total 40.12 last 16.16)
 - mkdir 708 (time 1790794001.50 total 50.15 last 15.56)
 - mkdir 871 (time 1790794011.53 total 60.17 last 16.26)
total: 1000 mkdir in 69.85 seconds: 14.32 ops/second
 - mkdir 126 (time 1790793961.89 total 10.11 last 12.47)
 - mkdir 257 (time 1790793971.96 total 20.18 last 13.01)
 - mkdir 381 (time 1790793981.97 total 30.18 last 12.40)
 - mkdir 546 (time 1790793991.99 total 40.21 last 16.45)
 - mkdir 710 (time 1790794002.03 total 50.24 last 16.34)
 - mkdir 867 (time 1790794012.08 total 60.29 last 15.63)
 - mkdir 996 (time 1790794022.14 total 70.35 last 12.83)
total: 1000 mkdir in 70.57 seconds: 14.17 ops/second
Time for file creation 82 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 16 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.42 seconds: 23.91 ops/second
 - mkdir 178 (time 1790794569.22 total 10.01 last 17.77)
 - mkdir 345 (time 1790794579.25 total 20.05 last 16.64)
 - mkdir 489 (time 1790794589.26 total 30.06 last 14.39)
 - mkdir 633 (time 1790794599.30 total 40.10 last 14.34)
 - mkdir 786 (time 1790794609.33 total 50.13 last 15.26)
 - mkdir 928 (time 1790794619.34 total 60.14 last 14.18)
total: 1000 mkdir in 64.63 seconds: 15.47 ops/second
 - mkdir 187 (time 1790794570.70 total 10.04 last 18.63)
 - mkdir 352 (time 1790794580.74 total 20.08 last 16.43)
 - mkdir 504 (time 1790794590.79 total 30.12 last 15.14)
 - mkdir 645 (time 1790794600.85 total 40.19 last 14.00)
 - mkdir 795 (time 1790794610.90 total 50.23 last 14.94)
 - mkdir 947 (time 1790794620.90 total 60.24 last 15.20)
total: 1000 mkdir in 64.30 seconds: 15.55 ops/second
 - mkdir 189 (time 1790794570.57 total 10.07 last 18.77)
 - mkdir 348 (time 1790794580.59 total 20.09 last 15.87)
 - mkdir 499 (time 1790794590.61 total 30.10 last 15.07)
 - mkdir 640 (time 1790794600.62 total 40.12 last 14.08)
 - mkdir 793 (time 1790794610.66 total 50.15 last 15.25)
 - mkdir 939 (time 1790794620.67 total 60.17 last 14.58)
total: 1000 mkdir in 64.77 seconds: 15.44 ops/second
 - mkdir 189 (time 1790794570.48 total 10.01 last 18.87)
 - mkdir 346 (time 1790794580.54 total 20.07 last 15.61)
 - mkdir 499 (time 1790794590.57 total 30.10 last 15.26)
 - mkdir 643 (time 1790794600.65 total 40.18 last 14.28)
 - mkdir 791 (time 1790794610.82 total 50.35 last 14.55)
 - mkdir 937 (time 1790794620.84 total 60.38 last 14.56)
total: 1000 mkdir in 65.27 seconds: 15.32 ops/second
 - mkdir 192 (time 1790794570.96 total 10.03 last 19.14)
 - mkdir 343 (time 1790794580.97 total 20.04 last 15.09)
 - mkdir 494 (time 1790794591.04 total 30.11 last 15.00)
 - mkdir 638 (time 1790794601.07 total 40.14 last 14.36)
 - mkdir 787 (time 1790794611.08 total 50.14 last 14.89)
 - mkdir 940 (time 1790794621.15 total 60.22 last 15.19)
total: 1000 mkdir in 64.95 seconds: 15.40 ops/second
 - mkdir 179 (time 1790794570.24 total 10.01 last 17.89)
 - mkdir 336 (time 1790794580.27 total 20.04 last 15.65)
 - mkdir 482 (time 1790794590.29 total 30.06 last 14.57)
 - mkdir 625 (time 1790794600.32 total 40.09 last 14.26)
 - mkdir 777 (time 1790794610.41 total 50.18 last 15.07)
 - mkdir 929 (time 1790794620.45 total 60.22 last 15.14)
total: 1000 mkdir in 65.76 seconds: 15.21 ops/second
 - mkdir 180 (time 1790794570.46 total 10.01 last 17.98)
 - mkdir 333 (time 1790794580.48 total 20.03 last 15.27)
 - mkdir 480 (time 1790794590.50 total 30.04 last 14.68)
 - mkdir 626 (time 1790794600.50 total 40.05 last 14.59)
 - mkdir 775 (time 1790794610.56 total 50.11 last 14.81)
 - mkdir 929 (time 1790794620.60 total 60.15 last 15.35)
total: 1000 mkdir in 65.62 seconds: 15.24 ops/second
 - mkdir 174 (time 1790794569.78 total 10.04 last 17.33)
 - mkdir 328 (time 1790794579.82 total 20.08 last 15.34)
 - mkdir 476 (time 1790794589.93 total 30.19 last 14.64)
 - mkdir 616 (time 1790794599.98 total 40.24 last 13.93)
 - mkdir 770 (time 1790794610.09 total 50.35 last 15.23)
 - mkdir 919 (time 1790794620.12 total 60.38 last 14.86)
total: 1000 mkdir in 67.11 seconds: 14.90 ops/second
 - mkdir 182 (time 1790794570.65 total 10.03 last 18.15)
 - mkdir 340 (time 1790794580.68 total 20.05 last 15.76)
 - mkdir 491 (time 1790794590.73 total 30.11 last 15.01)
 - mkdir 631 (time 1790794600.79 total 40.16 last 13.93)
 - mkdir 774 (time 1790794610.85 total 50.22 last 14.22)
 - mkdir 925 (time 1790794620.86 total 60.23 last 15.08)
total: 1000 mkdir in 66.23 seconds: 15.10 ops/second
 - mkdir 175 (time 1790794570.02 total 10.06 last 17.40)
 - mkdir 324 (time 1790794580.11 total 20.15 last 14.77)
 - mkdir 467 (time 1790794590.20 total 30.24 last 14.18)
 - mkdir 610 (time 1790794600.25 total 40.28 last 14.23)
 - mkdir 759 (time 1790794610.33 total 50.36 last 14.78)
 - mkdir 911 (time 1790794620.40 total 60.44 last 15.09)
total: 1000 mkdir in 67.91 seconds: 14.72 ops/second
Time for file creation 78 sec for 1 RPC sent at a time
 - unlinked 0 (time 1790794636 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1790794636 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1790794637 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1790794637 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1790794638 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1790794637 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1790794638 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1790794638 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1790794638 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1790794638 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
Time for file removal 42 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8