== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 15:08:55 (1789153735)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.70 seconds: 14.25 ops/second
 - mkdir 142 (time 1789153756.28 total 10.03 last 14.16)
 - mkdir 298 (time 1789153766.29 total 20.03 last 15.59)
 - mkdir 399 (time 1789153776.35 total 30.09 last 10.04)
 - mkdir 509 (time 1789153786.35 total 40.09 last 11.00)
 - mkdir 667 (time 1789153796.47 total 50.21 last 15.62)
 - mkdir 816 (time 1789153806.52 total 60.26 last 14.82)
 - mkdir 975 (time 1789153816.53 total 70.27 last 15.89)
total: 1000 mkdir in 72.78 seconds: 13.74 ops/second
 - mkdir 156 (time 1789153759.92 total 10.05 last 15.52)
 - mkdir 287 (time 1789153770.02 total 20.15 last 12.97)
 - mkdir 379 (time 1789153780.07 total 30.20 last 9.16)
 - mkdir 528 (time 1789153790.08 total 40.21 last 14.89)
 - mkdir 670 (time 1789153800.12 total 50.25 last 14.14)
 - mkdir 833 (time 1789153810.13 total 60.26 last 16.29)
 - mkdir 965 (time 1789153820.18 total 70.31 last 13.13)
total: 1000 mkdir in 75.30 seconds: 13.28 ops/second
 - mkdir 166 (time 1789153761.09 total 10.03 last 16.55)
 - mkdir 282 (time 1789153771.17 total 20.11 last 11.51)
 - mkdir 379 (time 1789153781.23 total 30.16 last 9.65)
 - mkdir 544 (time 1789153791.25 total 40.19 last 16.46)
 - mkdir 680 (time 1789153801.27 total 50.21 last 13.57)
 - mkdir 847 (time 1789153811.30 total 60.24 last 16.65)
 - mkdir 975 (time 1789153821.34 total 70.28 last 12.75)
total: 1000 mkdir in 74.35 seconds: 13.45 ops/second
 - mkdir 163 (time 1789153760.59 total 10.03 last 16.25)
 - mkdir 285 (time 1789153770.71 total 20.14 last 12.06)
 - mkdir 383 (time 1789153780.77 total 30.21 last 9.74)
 - mkdir 532 (time 1789153790.77 total 40.21 last 14.90)
 - mkdir 673 (time 1789153800.85 total 50.29 last 13.99)
 - mkdir 835 (time 1789153810.87 total 60.31 last 16.17)
 - mkdir 963 (time 1789153820.91 total 70.34 last 12.76)
total: 1000 mkdir in 76.38 seconds: 13.09 ops/second
 - mkdir 161 (time 1789153760.38 total 10.06 last 16.00)
 - mkdir 286 (time 1789153770.41 total 20.08 last 12.47)
 - mkdir 384 (time 1789153780.50 total 30.18 last 9.71)
 - mkdir 533 (time 1789153790.54 total 40.22 last 14.83)
 - mkdir 674 (time 1789153800.57 total 50.25 last 14.06)
 - mkdir 836 (time 1789153810.66 total 60.33 last 16.07)
 - mkdir 958 (time 1789153820.71 total 70.39 last 12.13)
total: 1000 mkdir in 76.81 seconds: 13.02 ops/second
 - mkdir 161 (time 1789153760.70 total 10.03 last 16.05)
 - mkdir 280 (time 1789153770.82 total 20.15 last 11.76)
 - mkdir 377 (time 1789153780.89 total 30.23 last 9.63)
 - mkdir 525 (time 1789153790.92 total 40.26 last 14.76)
 - mkdir 658 (time 1789153800.94 total 50.28 last 13.27)
 - mkdir 821 (time 1789153810.95 total 60.29 last 16.28)
 - mkdir 953 (time 1789153820.98 total 70.31 last 13.17)
total: 1000 mkdir in 77.53 seconds: 12.90 ops/second
 - mkdir 162 (time 1789153759.91 total 10.01 last 16.19)
 - mkdir 279 (time 1789153769.95 total 20.05 last 11.65)
 - mkdir 370 (time 1789153779.97 total 30.07 last 9.08)
 - mkdir 512 (time 1789153790.00 total 40.09 last 14.17)
 - mkdir 658 (time 1789153800.05 total 50.15 last 14.52)
 - mkdir 818 (time 1789153810.14 total 60.23 last 15.87)
 - mkdir 946 (time 1789153820.18 total 70.27 last 12.75)
total: 1000 mkdir in 78.46 seconds: 12.75 ops/second
 - mkdir 166 (time 1789153760.59 total 10.01 last 16.59)
 - mkdir 283 (time 1789153770.67 total 20.09 last 11.61)
 - mkdir 381 (time 1789153780.78 total 30.20 last 9.70)
 - mkdir 528 (time 1789153790.82 total 40.24 last 14.63)
 - mkdir 667 (time 1789153800.84 total 50.26 last 13.87)
 - mkdir 823 (time 1789153810.90 total 60.32 last 15.50)
 - mkdir 947 (time 1789153820.97 total 70.39 last 12.32)
total: 1000 mkdir in 78.20 seconds: 12.79 ops/second
 - mkdir 166 (time 1789153761.08 total 10.02 last 16.56)
 - mkdir 282 (time 1789153771.14 total 20.08 last 11.54)
 - mkdir 375 (time 1789153781.18 total 30.12 last 9.26)
 - mkdir 526 (time 1789153791.22 total 40.16 last 15.04)
 - mkdir 662 (time 1789153801.24 total 50.18 last 13.58)
 - mkdir 825 (time 1789153811.27 total 60.21 last 16.25)
 - mkdir 948 (time 1789153821.36 total 70.31 last 12.18)
total: 1000 mkdir in 78.43 seconds: 12.75 ops/second
 - mkdir 151 (time 1789153760.14 total 10.04 last 15.05)
 - mkdir 269 (time 1789153770.24 total 20.14 last 11.68)
 - mkdir 356 (time 1789153780.25 total 30.15 last 8.69)
 - mkdir 498 (time 1789153790.29 total 40.18 last 14.15)
 - mkdir 628 (time 1789153800.38 total 50.28 last 12.88)
 - mkdir 784 (time 1789153810.40 total 60.30 last 15.57)
 - mkdir 906 (time 1789153820.48 total 70.37 last 12.11)
 - mkdir 976 (time 1789153830.53 total 80.42 last 6.97)
total: 1000 mkdir in 81.83 seconds: 12.22 ops/second
Time for file creation 94 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.59 seconds: 16.87 ops/second
 - mkdir 180 (time 1789154383.57 total 10.05 last 17.91)
 - mkdir 347 (time 1789154393.61 total 20.09 last 16.62)
 - mkdir 527 (time 1789154403.63 total 30.11 last 17.96)
 - mkdir 783 (time 1789154413.66 total 40.14 last 25.54)
 - mkdir 959 (time 1789154423.73 total 50.21 last 17.47)
total: 1000 mkdir in 52.52 seconds: 19.04 ops/second
 - mkdir 196 (time 1789154385.09 total 10.02 last 19.55)
 - mkdir 364 (time 1789154395.15 total 20.08 last 16.71)
 - mkdir 566 (time 1789154405.17 total 30.10 last 20.15)
 - mkdir 813 (time 1789154415.23 total 40.16 last 24.57)
 - mkdir 972 (time 1789154425.24 total 50.17 last 15.88)
total: 1000 mkdir in 52.13 seconds: 19.18 ops/second
 - mkdir 192 (time 1789154384.71 total 10.01 last 19.19)
 - mkdir 358 (time 1789154394.77 total 20.07 last 16.49)
 - mkdir 544 (time 1789154404.78 total 30.08 last 18.59)
 - mkdir 799 (time 1789154414.84 total 40.13 last 25.35)
 - mkdir 961 (time 1789154424.85 total 50.15 last 16.18)
total: 1000 mkdir in 53.08 seconds: 18.84 ops/second
 - mkdir 184 (time 1789154384.78 total 10.02 last 18.36)
 - mkdir 349 (time 1789154394.79 total 20.04 last 16.48)
 - mkdir 538 (time 1789154404.83 total 30.08 last 18.82)
 - mkdir 789 (time 1789154414.84 total 40.09 last 25.09)
 - mkdir 951 (time 1789154424.86 total 50.11 last 16.17)
total: 1000 mkdir in 54.06 seconds: 18.50 ops/second
 - mkdir 191 (time 1789154385.51 total 10.01 last 19.08)
 - mkdir 352 (time 1789154395.55 total 20.06 last 16.03)
 - mkdir 550 (time 1789154405.63 total 30.13 last 19.65)
 - mkdir 801 (time 1789154415.68 total 40.18 last 24.98)
 - mkdir 962 (time 1789154425.72 total 50.23 last 16.02)
total: 1000 mkdir in 53.82 seconds: 18.58 ops/second
 - mkdir 183 (time 1789154384.78 total 10.00 last 18.30)
 - mkdir 343 (time 1789154394.87 total 20.09 last 15.87)
 - mkdir 538 (time 1789154404.88 total 30.10 last 19.47)
 - mkdir 782 (time 1789154414.90 total 40.11 last 24.37)
 - mkdir 947 (time 1789154424.91 total 50.13 last 16.48)
total: 1000 mkdir in 54.65 seconds: 18.30 ops/second
 - mkdir 183 (time 1789154384.35 total 10.04 last 18.23)
 - mkdir 338 (time 1789154394.51 total 20.21 last 15.25)
 - mkdir 518 (time 1789154404.54 total 30.23 last 17.95)
 - mkdir 777 (time 1789154414.55 total 40.24 last 25.87)
 - mkdir 938 (time 1789154424.59 total 50.28 last 16.03)
total: 1000 mkdir in 55.83 seconds: 17.91 ops/second
 - mkdir 186 (time 1789154385.42 total 10.04 last 18.52)
 - mkdir 347 (time 1789154395.44 total 20.06 last 16.07)
 - mkdir 541 (time 1789154405.46 total 30.08 last 19.36)
 - mkdir 790 (time 1789154415.54 total 40.17 last 24.70)
 - mkdir 949 (time 1789154425.54 total 50.17 last 15.90)
total: 1000 mkdir in 55.04 seconds: 18.17 ops/second
 - mkdir 184 (time 1789154385.05 total 10.05 last 18.31)
 - mkdir 345 (time 1789154395.08 total 20.08 last 16.06)
 - mkdir 528 (time 1789154405.08 total 30.08 last 18.30)
 - mkdir 775 (time 1789154415.11 total 40.11 last 24.62)
 - mkdir 932 (time 1789154425.14 total 50.14 last 15.66)
total: 1000 mkdir in 56.18 seconds: 17.80 ops/second
 - mkdir 157 (time 1789154383.04 total 10.02 last 15.67)
 - mkdir 316 (time 1789154393.09 total 20.07 last 15.82)
 - mkdir 478 (time 1789154403.09 total 30.07 last 16.20)
 - mkdir 719 (time 1789154413.10 total 40.07 last 24.08)
 - mkdir 903 (time 1789154423.12 total 50.09 last 18.36)
total: 1000 mkdir in 58.16 seconds: 17.19 ops/second
Time for file creation 68 sec for 1 RPC sent at a time
 - unlinked 0 (time 1789154439 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1789154441 ; total 1 ; last 1)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1789154442 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1789154443 ; total 1 ; last 1)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1789154442 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1789154442 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1789154442 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1789154443 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1789154442 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1789154443 ; total 1 ; last 1)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
Time for file removal 46 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8