== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 04:04:31 (1787904271)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.49 seconds: 20.37 ops/second
 - mkdir 144 (time 1787904291.84 total 10.02 last 14.37)
 - mkdir 285 (time 1787904301.86 total 20.05 last 14.07)
 - mkdir 422 (time 1787904311.89 total 30.07 last 13.67)
 - mkdir 563 (time 1787904321.90 total 40.09 last 14.08)
 - mkdir 719 (time 1787904331.94 total 50.13 last 15.54)
 - mkdir 870 (time 1787904341.98 total 60.16 last 15.05)
total: 1000 mkdir in 67.76 seconds: 14.76 ops/second
 - mkdir 139 (time 1787904292.35 total 10.04 last 13.84)
 - mkdir 279 (time 1787904302.35 total 20.05 last 13.99)
 - mkdir 414 (time 1787904312.38 total 30.08 last 13.46)
 - mkdir 560 (time 1787904322.41 total 40.11 last 14.57)
 - mkdir 715 (time 1787904332.49 total 50.19 last 15.38)
 - mkdir 872 (time 1787904342.56 total 60.26 last 15.58)
total: 1000 mkdir in 68.31 seconds: 14.64 ops/second
 - mkdir 139 (time 1787904292.87 total 10.05 last 13.83)
 - mkdir 281 (time 1787904302.90 total 20.08 last 14.16)
 - mkdir 414 (time 1787904312.91 total 30.09 last 13.29)
 - mkdir 557 (time 1787904323.01 total 40.19 last 14.16)
 - mkdir 717 (time 1787904333.04 total 50.22 last 15.96)
 - mkdir 870 (time 1787904343.09 total 60.27 last 15.22)
total: 1000 mkdir in 68.49 seconds: 14.60 ops/second
 - mkdir 148 (time 1787904294.01 total 10.05 last 14.72)
 - mkdir 289 (time 1787904304.04 total 20.09 last 14.06)
 - mkdir 429 (time 1787904314.07 total 30.12 last 13.95)
 - mkdir 569 (time 1787904324.11 total 40.15 last 13.95)
 - mkdir 724 (time 1787904334.14 total 50.18 last 15.45)
 - mkdir 886 (time 1787904344.14 total 60.19 last 16.19)
total: 1000 mkdir in 67.75 seconds: 14.76 ops/second
 - mkdir 153 (time 1787904294.16 total 10.11 last 15.13)
 - mkdir 301 (time 1787904304.19 total 20.14 last 14.76)
 - mkdir 439 (time 1787904314.20 total 30.15 last 13.78)
 - mkdir 578 (time 1787904324.21 total 40.16 last 13.89)
 - mkdir 729 (time 1787904334.27 total 50.22 last 15.01)
 - mkdir 894 (time 1787904344.31 total 60.26 last 16.44)
total: 1000 mkdir in 67.99 seconds: 14.71 ops/second
 - mkdir 141 (time 1787904293.71 total 10.03 last 14.05)
 - mkdir 282 (time 1787904303.72 total 20.04 last 14.09)
 - mkdir 417 (time 1787904313.76 total 30.08 last 13.44)
 - mkdir 557 (time 1787904323.82 total 40.14 last 13.91)
 - mkdir 721 (time 1787904333.86 total 50.18 last 16.33)
 - mkdir 878 (time 1787904343.90 total 60.22 last 15.64)
total: 1000 mkdir in 69.04 seconds: 14.48 ops/second
 - mkdir 137 (time 1787904293.29 total 10.07 last 13.61)
 - mkdir 279 (time 1787904303.33 total 20.12 last 14.13)
 - mkdir 418 (time 1787904313.36 total 30.14 last 13.86)
 - mkdir 560 (time 1787904323.39 total 40.17 last 14.16)
 - mkdir 718 (time 1787904333.49 total 50.27 last 15.65)
 - mkdir 870 (time 1787904343.52 total 60.30 last 15.15)
total: 1000 mkdir in 69.70 seconds: 14.35 ops/second
 - mkdir 144 (time 1787904294.00 total 10.03 last 14.36)
 - mkdir 287 (time 1787904304.03 total 20.06 last 14.26)
 - mkdir 421 (time 1787904314.06 total 30.09 last 13.36)
 - mkdir 564 (time 1787904324.09 total 40.12 last 14.25)
 - mkdir 721 (time 1787904334.12 total 50.15 last 15.65)
 - mkdir 881 (time 1787904344.14 total 60.17 last 15.98)
total: 1000 mkdir in 69.05 seconds: 14.48 ops/second
 - mkdir 137 (time 1787904294.02 total 10.03 last 13.66)
 - mkdir 276 (time 1787904304.05 total 20.07 last 13.85)
 - mkdir 410 (time 1787904314.06 total 30.07 last 13.40)
 - mkdir 548 (time 1787904324.07 total 40.08 last 13.78)
 - mkdir 706 (time 1787904334.09 total 50.10 last 15.78)
 - mkdir 866 (time 1787904344.12 total 60.14 last 15.94)
 - mkdir 994 (time 1787904354.13 total 70.14 last 12.79)
total: 1000 mkdir in 70.52 seconds: 14.18 ops/second
 - mkdir 130 (time 1787904293.47 total 10.05 last 12.94)
 - mkdir 268 (time 1787904303.49 total 20.06 last 13.78)
 - mkdir 398 (time 1787904313.52 total 30.10 last 12.96)
 - mkdir 538 (time 1787904323.56 total 40.14 last 13.94)
 - mkdir 699 (time 1787904333.65 total 50.23 last 15.96)
 - mkdir 854 (time 1787904343.70 total 60.28 last 15.42)
 - mkdir 983 (time 1787904353.74 total 70.32 last 12.85)
total: 1000 mkdir in 71.56 seconds: 13.97 ops/second
Time for file creation 81 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 10 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.42 seconds: 23.80 ops/second
 - mkdir 196 (time 1787904894.13 total 10.03 last 19.55)
 - mkdir 367 (time 1787904904.19 total 20.08 last 17.01)
 - mkdir 574 (time 1787904914.29 total 30.19 last 20.48)
 - mkdir 749 (time 1787904924.34 total 40.23 last 17.43)
 - mkdir 925 (time 1787904934.35 total 50.24 last 17.57)
total: 1000 mkdir in 54.78 seconds: 18.25 ops/second
 - mkdir 212 (time 1787904895.16 total 10.04 last 21.12)
 - mkdir 383 (time 1787904905.19 total 20.07 last 17.04)
 - mkdir 581 (time 1787904915.21 total 30.09 last 19.76)
 - mkdir 761 (time 1787904925.30 total 40.18 last 17.84)
 - mkdir 939 (time 1787904935.34 total 50.23 last 17.72)
total: 1000 mkdir in 54.07 seconds: 18.50 ops/second
 - mkdir 201 (time 1787904894.78 total 10.02 last 20.06)
 - mkdir 366 (time 1787904904.82 total 20.06 last 16.43)
 - mkdir 572 (time 1787904914.82 total 30.07 last 20.59)
 - mkdir 746 (time 1787904924.83 total 40.08 last 17.39)
 - mkdir 921 (time 1787904934.83 total 50.08 last 17.50)
total: 1000 mkdir in 55.36 seconds: 18.06 ops/second
 - mkdir 196 (time 1787904895.56 total 10.03 last 19.54)
 - mkdir 372 (time 1787904905.64 total 20.11 last 17.46)
 - mkdir 580 (time 1787904915.67 total 30.14 last 20.74)
 - mkdir 755 (time 1787904925.75 total 40.22 last 17.36)
 - mkdir 936 (time 1787904935.78 total 50.25 last 18.05)
total: 1000 mkdir in 54.87 seconds: 18.23 ops/second
 - mkdir 199 (time 1787904895.51 total 10.08 last 19.74)
 - mkdir 369 (time 1787904905.55 total 20.12 last 16.94)
 - mkdir 568 (time 1787904915.60 total 30.17 last 19.79)
 - mkdir 752 (time 1787904925.68 total 40.25 last 18.25)
 - mkdir 934 (time 1787904935.73 total 50.30 last 18.12)
total: 1000 mkdir in 55.00 seconds: 18.18 ops/second
 - mkdir 207 (time 1787904895.18 total 10.07 last 20.56)
 - mkdir 372 (time 1787904905.19 total 20.08 last 16.48)
 - mkdir 573 (time 1787904915.23 total 30.12 last 20.02)
 - mkdir 751 (time 1787904925.25 total 40.14 last 17.76)
 - mkdir 928 (time 1787904935.28 total 50.17 last 17.66)
total: 1000 mkdir in 55.33 seconds: 18.07 ops/second
 - mkdir 198 (time 1787904895.31 total 10.06 last 19.69)
 - mkdir 369 (time 1787904905.36 total 20.10 last 17.02)
 - mkdir 570 (time 1787904915.38 total 30.12 last 20.06)
 - mkdir 748 (time 1787904925.43 total 40.17 last 17.71)
 - mkdir 924 (time 1787904935.45 total 50.19 last 17.57)
total: 1000 mkdir in 55.41 seconds: 18.05 ops/second
 - mkdir 194 (time 1787904895.05 total 10.02 last 19.37)
 - mkdir 356 (time 1787904905.07 total 20.03 last 16.18)
 - mkdir 564 (time 1787904915.10 total 30.06 last 20.74)
 - mkdir 741 (time 1787904925.11 total 40.08 last 17.67)
 - mkdir 915 (time 1787904935.16 total 50.13 last 17.31)
total: 1000 mkdir in 56.13 seconds: 17.82 ops/second
 - mkdir 182 (time 1787904893.98 total 10.02 last 18.16)
 - mkdir 338 (time 1787904904.01 total 20.05 last 15.57)
 - mkdir 539 (time 1787904914.03 total 30.07 last 20.05)
 - mkdir 710 (time 1787904924.05 total 40.08 last 17.08)
 - mkdir 890 (time 1787904934.05 total 50.09 last 17.99)
total: 1000 mkdir in 57.88 seconds: 17.28 ops/second
 - mkdir 195 (time 1787904895.54 total 10.03 last 19.45)
 - mkdir 361 (time 1787904905.63 total 20.12 last 16.44)
 - mkdir 559 (time 1787904915.69 total 30.17 last 19.70)
 - mkdir 738 (time 1787904925.78 total 40.27 last 17.73)
 - mkdir 917 (time 1787904935.81 total 50.30 last 17.85)
total: 1000 mkdir in 56.35 seconds: 17.74 ops/second
Time for file creation 67 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787904950 ; total 0 ; last 0)
total: 1000 unlinks in 27 seconds: 37.037037 unlinks/second
 - unlinked 0 (time 1787904950 ; total 0 ; last 0)
total: 1000 unlinks in 27 seconds: 37.037037 unlinks/second
 - unlinked 0 (time 1787904951 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1787904951 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1787904951 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1787904951 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1787904951 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1787904952 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1787904952 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1787904951 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
Time for file removal 39 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8