== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 09:52:33 (1788961953)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.60 seconds: 16.66 ops/second
 - mkdir 122 (time 1788961973.15 total 10.05 last 12.13)
 - mkdir 278 (time 1788961983.18 total 20.09 last 15.55)
 - mkdir 450 (time 1788961993.21 total 30.11 last 17.15)
 - mkdir 615 (time 1788962003.26 total 40.17 last 16.41)
 - mkdir 792 (time 1788962013.34 total 50.24 last 17.57)
 - mkdir 947 (time 1788962023.35 total 60.25 last 15.48)
total: 1000 mkdir in 63.63 seconds: 15.71 ops/second
 - mkdir 140 (time 1788961979.36 total 10.03 last 13.96)
 - mkdir 300 (time 1788961989.37 total 20.04 last 15.98)
 - mkdir 490 (time 1788961999.43 total 30.10 last 18.88)
 - mkdir 678 (time 1788962009.47 total 40.14 last 18.73)
 - mkdir 837 (time 1788962019.47 total 50.14 last 15.89)
 - mkdir 963 (time 1788962029.52 total 60.19 last 12.54)
total: 1000 mkdir in 65.75 seconds: 15.21 ops/second
 - mkdir 133 (time 1788961978.40 total 10.01 last 13.28)
 - mkdir 290 (time 1788961988.44 total 20.04 last 15.65)
 - mkdir 481 (time 1788961998.48 total 30.09 last 19.02)
 - mkdir 660 (time 1788962008.53 total 40.14 last 17.81)
 - mkdir 824 (time 1788962018.54 total 50.15 last 16.38)
 - mkdir 949 (time 1788962028.62 total 60.23 last 12.40)
total: 1000 mkdir in 66.89 seconds: 14.95 ops/second
 - mkdir 135 (time 1788961978.10 total 10.01 last 13.48)
 - mkdir 284 (time 1788961988.16 total 20.08 last 14.81)
 - mkdir 474 (time 1788961998.28 total 30.20 last 18.77)
 - mkdir 651 (time 1788962008.31 total 40.22 last 17.66)
 - mkdir 820 (time 1788962018.32 total 50.24 last 16.87)
 - mkdir 948 (time 1788962028.48 total 60.39 last 12.60)
total: 1000 mkdir in 67.51 seconds: 14.81 ops/second
 - mkdir 140 (time 1788961979.37 total 10.02 last 13.97)
 - mkdir 300 (time 1788961989.44 total 20.10 last 15.88)
 - mkdir 487 (time 1788961999.52 total 30.18 last 18.56)
 - mkdir 685 (time 1788962009.54 total 40.19 last 19.77)
 - mkdir 836 (time 1788962019.58 total 50.24 last 15.03)
 - mkdir 958 (time 1788962029.63 total 60.29 last 12.14)
total: 1000 mkdir in 66.44 seconds: 15.05 ops/second
 - mkdir 140 (time 1788961978.96 total 10.01 last 13.99)
 - mkdir 291 (time 1788961989.01 total 20.06 last 15.03)
 - mkdir 477 (time 1788961999.03 total 30.08 last 18.57)
 - mkdir 663 (time 1788962009.04 total 40.09 last 18.57)
 - mkdir 828 (time 1788962019.09 total 50.14 last 16.42)
 - mkdir 951 (time 1788962029.21 total 60.25 last 12.16)
total: 1000 mkdir in 67.60 seconds: 14.79 ops/second
 - mkdir 138 (time 1788961979.07 total 10.00 last 13.80)
 - mkdir 295 (time 1788961989.12 total 20.06 last 15.62)
 - mkdir 479 (time 1788961999.14 total 30.08 last 18.36)
 - mkdir 666 (time 1788962009.15 total 40.09 last 18.67)
 - mkdir 823 (time 1788962019.22 total 50.16 last 15.60)
 - mkdir 944 (time 1788962029.31 total 60.25 last 11.99)
total: 1000 mkdir in 68.42 seconds: 14.62 ops/second
 - mkdir 131 (time 1788961978.03 total 10.01 last 13.09)
 - mkdir 274 (time 1788961988.05 total 20.03 last 14.27)
 - mkdir 459 (time 1788961998.08 total 30.05 last 18.45)
 - mkdir 629 (time 1788962008.09 total 40.07 last 16.97)
 - mkdir 787 (time 1788962018.16 total 50.14 last 15.69)
 - mkdir 913 (time 1788962028.36 total 60.33 last 12.36)
 - mkdir 989 (time 1788962038.40 total 70.38 last 7.57)
total: 1000 mkdir in 71.55 seconds: 13.98 ops/second
 - mkdir 139 (time 1788961979.83 total 10.02 last 13.87)
 - mkdir 291 (time 1788961989.83 total 20.02 last 15.20)
 - mkdir 465 (time 1788961999.86 total 30.06 last 17.34)
 - mkdir 651 (time 1788962009.97 total 40.17 last 18.40)
 - mkdir 803 (time 1788962020.00 total 50.20 last 15.15)
 - mkdir 924 (time 1788962030.13 total 60.32 last 11.95)
 - mkdir 997 (time 1788962040.20 total 70.39 last 7.25)
total: 1000 mkdir in 70.51 seconds: 14.18 ops/second
 - mkdir 127 (time 1788961978.65 total 10.01 last 12.69)
 - mkdir 270 (time 1788961988.66 total 20.01 last 14.29)
 - mkdir 450 (time 1788961998.74 total 30.10 last 17.84)
 - mkdir 623 (time 1788962008.77 total 40.13 last 17.26)
 - mkdir 770 (time 1788962018.77 total 50.13 last 14.70)
 - mkdir 892 (time 1788962028.77 total 60.13 last 12.20)
 - mkdir 965 (time 1788962038.78 total 70.14 last 7.29)
total: 1000 mkdir in 72.93 seconds: 13.71 ops/second
Time for file creation 85 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 17 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.45 seconds: 22.14 ops/second
 - mkdir 164 (time 1788962549.57 total 10.03 last 16.35)
 - mkdir 336 (time 1788962559.57 total 20.04 last 17.19)
 - mkdir 542 (time 1788962569.63 total 30.10 last 20.48)
 - mkdir 710 (time 1788962579.73 total 40.20 last 16.63)
 - mkdir 862 (time 1788962589.78 total 50.25 last 15.12)
total: 1000 mkdir in 58.87 seconds: 16.99 ops/second
 - mkdir 170 (time 1788962551.09 total 10.07 last 16.88)
 - mkdir 337 (time 1788962561.15 total 20.13 last 16.60)
 - mkdir 559 (time 1788962571.17 total 30.16 last 22.14)
 - mkdir 720 (time 1788962581.19 total 40.17 last 16.07)
 - mkdir 874 (time 1788962591.24 total 50.23 last 15.32)
total: 1000 mkdir in 58.91 seconds: 16.97 ops/second
 - mkdir 167 (time 1788962550.70 total 10.03 last 16.64)
 - mkdir 330 (time 1788962560.74 total 20.07 last 16.24)
 - mkdir 544 (time 1788962570.76 total 30.09 last 21.35)
 - mkdir 706 (time 1788962580.77 total 40.11 last 16.18)
 - mkdir 864 (time 1788962590.80 total 50.14 last 15.75)
total: 1000 mkdir in 59.27 seconds: 16.87 ops/second
 - mkdir 172 (time 1788962550.94 total 10.01 last 17.18)
 - mkdir 334 (time 1788962560.95 total 20.03 last 16.17)
 - mkdir 550 (time 1788962570.97 total 30.04 last 21.56)
 - mkdir 711 (time 1788962580.99 total 40.07 last 16.06)
 - mkdir 864 (time 1788962591.00 total 50.07 last 15.30)
total: 1000 mkdir in 59.18 seconds: 16.90 ops/second
 - mkdir 173 (time 1788962551.86 total 10.06 last 17.19)
 - mkdir 356 (time 1788962561.87 total 20.08 last 18.28)
 - mkdir 564 (time 1788962571.89 total 30.09 last 20.76)
 - mkdir 723 (time 1788962581.90 total 40.10 last 15.88)
 - mkdir 881 (time 1788962591.91 total 50.11 last 15.78)
total: 1000 mkdir in 58.52 seconds: 17.09 ops/second
 - mkdir 171 (time 1788962551.85 total 10.06 last 17.00)
 - mkdir 352 (time 1788962561.88 total 20.09 last 18.05)
 - mkdir 565 (time 1788962571.90 total 30.10 last 21.26)
 - mkdir 724 (time 1788962582.01 total 40.22 last 15.72)
 - mkdir 881 (time 1788962592.01 total 50.22 last 15.70)
total: 1000 mkdir in 58.63 seconds: 17.06 ops/second
 - mkdir 175 (time 1788962551.44 total 10.08 last 17.35)
 - mkdir 340 (time 1788962561.46 total 20.11 last 16.46)
 - mkdir 557 (time 1788962571.56 total 30.21 last 21.48)
 - mkdir 717 (time 1788962581.60 total 40.24 last 15.94)
 - mkdir 873 (time 1788962591.68 total 50.32 last 15.48)
total: 1000 mkdir in 59.31 seconds: 16.86 ops/second
 - mkdir 166 (time 1788962551.37 total 10.06 last 16.50)
 - mkdir 329 (time 1788962561.37 total 20.06 last 16.30)
 - mkdir 544 (time 1788962571.39 total 30.08 last 21.46)
 - mkdir 702 (time 1788962581.44 total 40.13 last 15.71)
 - mkdir 858 (time 1788962591.48 total 50.17 last 15.54)
total: 1000 mkdir in 60.16 seconds: 16.62 ops/second
 - mkdir 164 (time 1788962551.28 total 10.07 last 16.29)
 - mkdir 322 (time 1788962561.29 total 20.07 last 15.79)
 - mkdir 542 (time 1788962571.36 total 30.14 last 21.85)
 - mkdir 704 (time 1788962581.38 total 40.17 last 16.16)
 - mkdir 859 (time 1788962591.42 total 50.21 last 15.44)
 - mkdir 996 (time 1788962601.44 total 60.23 last 13.68)
total: 1000 mkdir in 60.41 seconds: 16.55 ops/second
 - mkdir 157 (time 1788962551.67 total 10.06 last 15.60)
 - mkdir 320 (time 1788962561.69 total 20.08 last 16.27)
 - mkdir 520 (time 1788962571.71 total 30.10 last 19.96)
 - mkdir 677 (time 1788962581.75 total 40.15 last 15.63)
 - mkdir 833 (time 1788962591.81 total 50.20 last 15.51)
 - mkdir 974 (time 1788962601.85 total 60.24 last 14.04)
total: 1000 mkdir in 62.55 seconds: 15.99 ops/second
Time for file creation 74 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788962612 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788962614 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1788962614 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1788962615 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788962615 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788962615 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788962615 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788962615 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788962615 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788962615 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
Time for file removal 44 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8