== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 21:56:10 (1787882170)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.63 seconds: 15.84 ops/second
 - mkdir 157 (time 1787882190.83 total 10.04 last 15.64)
 - mkdir 308 (time 1787882200.86 total 20.08 last 15.05)
 - mkdir 465 (time 1787882210.86 total 30.08 last 15.70)
 - mkdir 617 (time 1787882220.93 total 40.14 last 15.11)
 - mkdir 755 (time 1787882230.94 total 50.16 last 13.78)
 - mkdir 891 (time 1787882241.00 total 60.22 last 13.52)
total: 1000 mkdir in 68.57 seconds: 14.58 ops/second
 - mkdir 169 (time 1787882193.00 total 10.09 last 16.75)
 - mkdir 320 (time 1787882203.05 total 20.15 last 15.01)
 - mkdir 484 (time 1787882213.11 total 30.20 last 16.31)
 - mkdir 634 (time 1787882223.17 total 40.26 last 14.92)
 - mkdir 771 (time 1787882233.17 total 50.26 last 13.69)
 - mkdir 900 (time 1787882243.19 total 60.28 last 12.88)
total: 1000 mkdir in 68.20 seconds: 14.66 ops/second
 - mkdir 168 (time 1787882193.33 total 10.04 last 16.74)
 - mkdir 319 (time 1787882203.36 total 20.07 last 15.04)
 - mkdir 481 (time 1787882213.39 total 30.10 last 16.16)
 - mkdir 633 (time 1787882223.41 total 40.12 last 15.17)
 - mkdir 768 (time 1787882233.42 total 50.12 last 13.49)
 - mkdir 900 (time 1787882243.48 total 60.19 last 13.11)
total: 1000 mkdir in 67.82 seconds: 14.74 ops/second
 - mkdir 150 (time 1787882190.07 total 10.00 last 15.00)
 - mkdir 290 (time 1787882200.12 total 20.06 last 13.92)
 - mkdir 445 (time 1787882210.22 total 30.16 last 15.34)
 - mkdir 595 (time 1787882220.25 total 40.19 last 14.96)
 - mkdir 735 (time 1787882230.35 total 50.29 last 13.86)
 - mkdir 870 (time 1787882240.39 total 60.33 last 13.44)
 - mkdir 988 (time 1787882250.41 total 70.35 last 11.78)
total: 1000 mkdir in 72.55 seconds: 13.78 ops/second
 - mkdir 163 (time 1787882193.00 total 10.12 last 16.10)
 - mkdir 307 (time 1787882203.03 total 20.16 last 14.35)
 - mkdir 470 (time 1787882213.06 total 30.19 last 16.25)
 - mkdir 623 (time 1787882223.13 total 40.25 last 15.20)
 - mkdir 759 (time 1787882233.21 total 50.34 last 13.49)
 - mkdir 891 (time 1787882243.23 total 60.35 last 13.18)
total: 1000 mkdir in 70.26 seconds: 14.23 ops/second
 - mkdir 156 (time 1787882192.04 total 10.04 last 15.54)
 - mkdir 294 (time 1787882202.06 total 20.05 last 13.78)
 - mkdir 448 (time 1787882212.07 total 30.07 last 15.38)
 - mkdir 601 (time 1787882222.10 total 40.10 last 15.26)
 - mkdir 738 (time 1787882232.10 total 50.10 last 13.69)
 - mkdir 874 (time 1787882242.21 total 60.20 last 13.46)
 - mkdir 994 (time 1787882252.44 total 70.44 last 11.72)
total: 1000 mkdir in 71.63 seconds: 13.96 ops/second
 - mkdir 160 (time 1787882192.68 total 10.04 last 15.93)
 - mkdir 301 (time 1787882202.70 total 20.06 last 14.07)
 - mkdir 463 (time 1787882212.70 total 30.07 last 16.19)
 - mkdir 615 (time 1787882222.73 total 40.10 last 15.15)
 - mkdir 750 (time 1787882232.77 total 50.14 last 13.45)
 - mkdir 878 (time 1787882242.82 total 60.19 last 12.74)
 - mkdir 991 (time 1787882252.83 total 70.20 last 11.28)
total: 1000 mkdir in 71.40 seconds: 14.00 ops/second
 - mkdir 163 (time 1787882192.90 total 10.06 last 16.20)
 - mkdir 307 (time 1787882202.93 total 20.09 last 14.36)
 - mkdir 470 (time 1787882212.98 total 30.14 last 16.22)
 - mkdir 622 (time 1787882223.05 total 40.20 last 15.10)
 - mkdir 753 (time 1787882233.08 total 50.24 last 13.05)
 - mkdir 885 (time 1787882243.12 total 60.28 last 13.15)
 - mkdir 992 (time 1787882253.18 total 70.34 last 10.64)
total: 1000 mkdir in 71.37 seconds: 14.01 ops/second
 - mkdir 155 (time 1787882192.45 total 10.01 last 15.48)
 - mkdir 298 (time 1787882202.47 total 20.03 last 14.28)
 - mkdir 459 (time 1787882212.53 total 30.09 last 16.01)
 - mkdir 613 (time 1787882222.56 total 40.12 last 15.36)
 - mkdir 749 (time 1787882232.56 total 50.12 last 13.60)
 - mkdir 877 (time 1787882242.61 total 60.17 last 12.74)
 - mkdir 991 (time 1787882252.77 total 70.33 last 11.22)
total: 1000 mkdir in 71.86 seconds: 13.92 ops/second
 - mkdir 169 (time 1787882193.00 total 10.09 last 16.75)
 - mkdir 312 (time 1787882203.00 total 20.09 last 14.29)
 - mkdir 469 (time 1787882213.05 total 30.14 last 15.63)
 - mkdir 614 (time 1787882223.13 total 40.22 last 14.38)
 - mkdir 749 (time 1787882233.25 total 50.34 last 13.33)
 - mkdir 884 (time 1787882243.38 total 60.47 last 13.34)
 - mkdir 988 (time 1787882253.57 total 70.66 last 10.20)
total: 1000 mkdir in 72.30 seconds: 13.83 ops/second
Time for file creation 85 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 17 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.67 seconds: 15.01 ops/second
 - mkdir 140 (time 1787882808.81 total 10.04 last 13.95)
 - mkdir 294 (time 1787882818.85 total 20.07 last 15.34)
 - mkdir 456 (time 1787882828.85 total 30.08 last 16.20)
 - mkdir 636 (time 1787882838.95 total 40.18 last 17.82)
 - mkdir 824 (time 1787882849.02 total 50.25 last 18.67)
total: 1000 mkdir in 60.01 seconds: 16.66 ops/second
 - mkdir 139 (time 1787882807.98 total 10.00 last 13.90)
 - mkdir 297 (time 1787882818.06 total 20.08 last 15.68)
 - mkdir 450 (time 1787882828.07 total 30.09 last 15.29)
 - mkdir 623 (time 1787882838.08 total 40.10 last 17.28)
 - mkdir 801 (time 1787882848.09 total 50.11 last 17.78)
 - mkdir 978 (time 1787882858.11 total 60.13 last 17.67)
total: 1000 mkdir in 61.82 seconds: 16.18 ops/second
 - mkdir 139 (time 1787882808.74 total 10.06 last 13.82)
 - mkdir 296 (time 1787882818.78 total 20.10 last 15.64)
 - mkdir 450 (time 1787882828.84 total 30.16 last 15.31)
 - mkdir 623 (time 1787882838.90 total 40.22 last 17.19)
 - mkdir 801 (time 1787882848.93 total 50.25 last 17.75)
 - mkdir 971 (time 1787882858.94 total 60.26 last 16.98)
total: 1000 mkdir in 62.92 seconds: 15.89 ops/second
 - mkdir 141 (time 1787882808.48 total 10.01 last 14.08)
 - mkdir 290 (time 1787882818.51 total 20.04 last 14.86)
 - mkdir 442 (time 1787882828.56 total 30.09 last 15.13)
 - mkdir 615 (time 1787882838.58 total 40.12 last 17.26)
 - mkdir 800 (time 1787882848.61 total 50.14 last 18.46)
 - mkdir 966 (time 1787882858.64 total 60.17 last 16.55)
total: 1000 mkdir in 63.46 seconds: 15.76 ops/second
 - mkdir 136 (time 1787882808.72 total 10.04 last 13.55)
 - mkdir 286 (time 1787882818.73 total 20.05 last 14.98)
 - mkdir 439 (time 1787882828.78 total 30.09 last 15.23)
 - mkdir 613 (time 1787882838.81 total 40.12 last 17.35)
 - mkdir 794 (time 1787882848.84 total 50.16 last 18.04)
 - mkdir 968 (time 1787882858.90 total 60.22 last 17.30)
total: 1000 mkdir in 63.81 seconds: 15.67 ops/second
 - mkdir 127 (time 1787882806.95 total 10.01 last 12.69)
 - mkdir 278 (time 1787882816.97 total 20.04 last 15.06)
 - mkdir 419 (time 1787882827.00 total 30.06 last 14.06)
 - mkdir 586 (time 1787882837.00 total 40.07 last 16.69)
 - mkdir 764 (time 1787882847.04 total 50.10 last 17.74)
 - mkdir 935 (time 1787882857.06 total 60.13 last 17.05)
total: 1000 mkdir in 65.58 seconds: 15.25 ops/second
 - mkdir 133 (time 1787882808.67 total 10.03 last 13.26)
 - mkdir 282 (time 1787882818.70 total 20.06 last 14.86)
 - mkdir 436 (time 1787882828.76 total 30.13 last 15.29)
 - mkdir 614 (time 1787882838.77 total 40.13 last 17.79)
 - mkdir 793 (time 1787882848.81 total 50.18 last 17.83)
 - mkdir 964 (time 1787882858.87 total 60.23 last 17.01)
total: 1000 mkdir in 63.98 seconds: 15.63 ops/second
 - mkdir 138 (time 1787882808.47 total 10.03 last 13.75)
 - mkdir 286 (time 1787882818.49 total 20.06 last 14.76)
 - mkdir 439 (time 1787882828.52 total 30.08 last 15.27)
 - mkdir 610 (time 1787882838.55 total 40.11 last 17.05)
 - mkdir 783 (time 1787882848.59 total 50.15 last 17.22)
 - mkdir 955 (time 1787882858.63 total 60.19 last 17.14)
total: 1000 mkdir in 64.39 seconds: 15.53 ops/second
 - mkdir 141 (time 1787882808.62 total 10.02 last 14.07)
 - mkdir 292 (time 1787882818.67 total 20.08 last 15.02)
 - mkdir 450 (time 1787882828.67 total 30.08 last 15.80)
 - mkdir 620 (time 1787882838.68 total 40.09 last 16.99)
 - mkdir 797 (time 1787882848.78 total 50.18 last 17.53)
 - mkdir 959 (time 1787882858.82 total 60.22 last 16.13)
total: 1000 mkdir in 64.39 seconds: 15.53 ops/second
 - mkdir 126 (time 1787882807.83 total 10.05 last 12.53)
 - mkdir 271 (time 1787882817.84 total 20.05 last 14.50)
 - mkdir 414 (time 1787882827.86 total 30.08 last 14.26)
 - mkdir 575 (time 1787882837.93 total 40.14 last 16.00)
 - mkdir 743 (time 1787882847.99 total 50.20 last 16.70)
 - mkdir 909 (time 1787882858.05 total 60.26 last 16.50)
total: 1000 mkdir in 67.86 seconds: 14.74 ops/second
Time for file creation 79 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787882873 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787882873 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787882873 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787882874 ; total 1 ; last 1)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787882874 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787882874 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787882874 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787882874 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787882874 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787882873 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
Time for file removal 45 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8