== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 14:04:41 (1787594681)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.58 seconds: 17.29 ops/second
 - mkdir 184 (time 1787594702.35 total 10.01 last 18.38)
 - mkdir 378 (time 1787594712.41 total 20.07 last 19.28)
 - mkdir 561 (time 1787594722.45 total 30.11 last 18.24)
 - mkdir 754 (time 1787594732.46 total 40.12 last 19.28)
 - mkdir 945 (time 1787594742.47 total 50.13 last 19.07)
total: 1000 mkdir in 53.03 seconds: 18.86 ops/second
 - mkdir 175 (time 1787594702.05 total 10.05 last 17.41)
 - mkdir 373 (time 1787594712.07 total 20.08 last 19.75)
 - mkdir 557 (time 1787594722.08 total 30.08 last 18.40)
 - mkdir 746 (time 1787594732.10 total 40.11 last 18.85)
 - mkdir 930 (time 1787594742.10 total 50.11 last 18.39)
total: 1000 mkdir in 54.23 seconds: 18.44 ops/second
 - mkdir 181 (time 1787594702.30 total 10.02 last 18.06)
 - mkdir 367 (time 1787594712.31 total 20.03 last 18.58)
 - mkdir 557 (time 1787594722.36 total 30.08 last 18.91)
 - mkdir 747 (time 1787594732.38 total 40.11 last 18.95)
 - mkdir 933 (time 1787594742.40 total 50.13 last 18.56)
total: 1000 mkdir in 54.16 seconds: 18.46 ops/second
 - mkdir 176 (time 1787594702.30 total 10.01 last 17.58)
 - mkdir 368 (time 1787594712.31 total 20.02 last 19.19)
 - mkdir 551 (time 1787594722.32 total 30.03 last 18.27)
 - mkdir 745 (time 1787594732.33 total 40.04 last 19.39)
 - mkdir 927 (time 1787594742.36 total 50.07 last 18.14)
total: 1000 mkdir in 54.37 seconds: 18.39 ops/second
 - mkdir 172 (time 1787594702.16 total 10.04 last 17.13)
 - mkdir 362 (time 1787594712.16 total 20.04 last 19.00)
 - mkdir 542 (time 1787594722.24 total 30.11 last 17.87)
 - mkdir 736 (time 1787594732.26 total 40.14 last 19.35)
 - mkdir 925 (time 1787594742.29 total 50.16 last 18.85)
total: 1000 mkdir in 54.66 seconds: 18.29 ops/second
 - mkdir 178 (time 1787594700.29 total 10.01 last 17.79)
 - mkdir 353 (time 1787594710.31 total 20.02 last 17.47)
 - mkdir 533 (time 1787594720.33 total 30.04 last 17.96)
 - mkdir 716 (time 1787594730.36 total 40.08 last 18.23)
 - mkdir 893 (time 1787594740.39 total 50.10 last 17.66)
total: 1000 mkdir in 57.06 seconds: 17.53 ops/second
 - mkdir 183 (time 1787594702.59 total 10.00 last 18.30)
 - mkdir 367 (time 1787594712.59 total 20.00 last 18.40)
 - mkdir 552 (time 1787594722.62 total 30.03 last 18.45)
 - mkdir 742 (time 1787594732.62 total 40.03 last 19.00)
 - mkdir 925 (time 1787594742.63 total 50.04 last 18.28)
total: 1000 mkdir in 54.95 seconds: 18.20 ops/second
 - mkdir 183 (time 1787594702.76 total 10.01 last 18.29)
 - mkdir 373 (time 1787594712.79 total 20.04 last 18.94)
 - mkdir 554 (time 1787594722.83 total 30.08 last 18.03)
 - mkdir 747 (time 1787594732.89 total 40.14 last 19.18)
 - mkdir 928 (time 1787594742.93 total 50.18 last 18.03)
total: 1000 mkdir in 54.89 seconds: 18.22 ops/second
 - mkdir 177 (time 1787594702.35 total 10.03 last 17.64)
 - mkdir 366 (time 1787594712.38 total 20.06 last 18.85)
 - mkdir 541 (time 1787594722.43 total 30.11 last 17.40)
 - mkdir 732 (time 1787594732.45 total 40.13 last 19.07)
 - mkdir 913 (time 1787594742.47 total 50.15 last 18.06)
total: 1000 mkdir in 56.05 seconds: 17.84 ops/second
 - mkdir 170 (time 1787594702.27 total 10.02 last 16.97)
 - mkdir 355 (time 1787594712.27 total 20.02 last 18.50)
 - mkdir 520 (time 1787594722.28 total 30.02 last 16.49)
 - mkdir 714 (time 1787594732.31 total 40.06 last 19.33)
 - mkdir 893 (time 1787594742.35 total 50.09 last 17.84)
total: 1000 mkdir in 57.73 seconds: 17.32 ops/second
Time for file creation 66 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.60 seconds: 16.71 ops/second
 - mkdir 101 (time 1787595265.53 total 10.04 last 10.06)
 - mkdir 262 (time 1787595275.55 total 20.06 last 16.07)
 - mkdir 409 (time 1787595285.56 total 30.08 last 14.68)
 - mkdir 566 (time 1787595295.60 total 40.12 last 15.64)
 - mkdir 724 (time 1787595305.61 total 50.12 last 15.80)
 - mkdir 878 (time 1787595315.64 total 60.15 last 15.35)
total: 1000 mkdir in 67.66 seconds: 14.78 ops/second
 - mkdir 103 (time 1787595265.52 total 10.09 last 10.21)
 - mkdir 262 (time 1787595275.74 total 20.30 last 15.57)
 - mkdir 410 (time 1787595285.76 total 30.33 last 14.76)
 - mkdir 562 (time 1787595295.78 total 40.34 last 15.17)
 - mkdir 721 (time 1787595305.83 total 50.40 last 15.82)
 - mkdir 877 (time 1787595315.93 total 60.50 last 15.44)
total: 1000 mkdir in 68.16 seconds: 14.67 ops/second
 - mkdir 107 (time 1787595266.12 total 10.06 last 10.64)
 - mkdir 259 (time 1787595276.14 total 20.07 last 15.17)
 - mkdir 410 (time 1787595286.15 total 30.09 last 15.08)
 - mkdir 562 (time 1787595296.27 total 40.20 last 15.03)
 - mkdir 724 (time 1787595306.29 total 50.22 last 16.17)
 - mkdir 876 (time 1787595316.29 total 60.22 last 15.20)
total: 1000 mkdir in 67.72 seconds: 14.77 ops/second
 - mkdir 103 (time 1787595265.81 total 10.10 last 10.20)
 - mkdir 258 (time 1787595275.86 total 20.16 last 15.42)
 - mkdir 405 (time 1787595285.90 total 30.19 last 14.64)
 - mkdir 551 (time 1787595295.91 total 40.21 last 14.58)
 - mkdir 709 (time 1787595305.93 total 50.22 last 15.77)
 - mkdir 869 (time 1787595315.94 total 60.24 last 15.98)
total: 1000 mkdir in 69.19 seconds: 14.45 ops/second
 - mkdir 97 (time 1787595265.63 total 10.03 last 9.67)
 - mkdir 255 (time 1787595275.74 total 20.14 last 15.63)
 - mkdir 403 (time 1787595285.78 total 30.19 last 14.74)
 - mkdir 547 (time 1787595295.80 total 40.21 last 14.37)
 - mkdir 706 (time 1787595305.83 total 50.23 last 15.86)
 - mkdir 856 (time 1787595315.92 total 60.33 last 14.86)
total: 1000 mkdir in 70.17 seconds: 14.25 ops/second
 - mkdir 96 (time 1787595265.15 total 10.04 last 9.56)
 - mkdir 253 (time 1787595275.18 total 20.06 last 15.66)
 - mkdir 392 (time 1787595285.20 total 30.09 last 13.87)
 - mkdir 545 (time 1787595295.33 total 40.22 last 15.11)
 - mkdir 702 (time 1787595305.34 total 50.23 last 15.68)
 - mkdir 853 (time 1787595315.35 total 60.23 last 15.10)
 - mkdir 996 (time 1787595325.49 total 70.38 last 14.10)
total: 1000 mkdir in 70.73 seconds: 14.14 ops/second
 - mkdir 106 (time 1787595266.62 total 10.10 last 10.50)
 - mkdir 259 (time 1787595276.66 total 20.13 last 15.25)
 - mkdir 409 (time 1787595286.67 total 30.15 last 14.98)
 - mkdir 551 (time 1787595296.74 total 40.22 last 14.11)
 - mkdir 703 (time 1787595306.75 total 50.22 last 15.19)
 - mkdir 854 (time 1787595316.77 total 60.24 last 15.07)
 - mkdir 993 (time 1787595326.81 total 70.28 last 13.84)
total: 1000 mkdir in 71.01 seconds: 14.08 ops/second
 - mkdir 101 (time 1787595266.08 total 10.09 last 10.01)
 - mkdir 254 (time 1787595276.15 total 20.16 last 15.19)
 - mkdir 401 (time 1787595286.15 total 30.16 last 14.70)
 - mkdir 548 (time 1787595296.25 total 40.26 last 14.56)
 - mkdir 704 (time 1787595306.27 total 50.28 last 15.57)
 - mkdir 854 (time 1787595316.30 total 60.31 last 14.95)
 - mkdir 986 (time 1787595326.31 total 70.32 last 13.19)
total: 1000 mkdir in 71.88 seconds: 13.91 ops/second
 - mkdir 104 (time 1787595265.53 total 10.08 last 10.32)
 - mkdir 258 (time 1787595275.68 total 20.23 last 15.17)
 - mkdir 395 (time 1787595285.69 total 30.24 last 13.68)
 - mkdir 539 (time 1787595295.72 total 40.27 last 14.36)
 - mkdir 693 (time 1787595305.74 total 50.29 last 15.38)
 - mkdir 839 (time 1787595315.84 total 60.38 last 14.46)
 - mkdir 972 (time 1787595325.84 total 70.39 last 13.30)
total: 1000 mkdir in 73.36 seconds: 13.63 ops/second
 - mkdir 95 (time 1787595264.18 total 10.02 last 9.48)
 - mkdir 238 (time 1787595274.22 total 20.06 last 14.24)
 - mkdir 377 (time 1787595284.22 total 30.06 last 13.90)
 - mkdir 529 (time 1787595294.26 total 40.11 last 15.14)
 - mkdir 672 (time 1787595304.27 total 50.11 last 14.29)
 - mkdir 809 (time 1787595314.31 total 60.16 last 13.64)
 - mkdir 957 (time 1787595324.32 total 70.16 last 14.79)
total: 1000 mkdir in 74.79 seconds: 13.37 ops/second
Time for file creation 83 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787595336 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1787595336 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1787595336 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787595337 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787595338 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1787595337 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787595337 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787595337 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787595338 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787595338 ; total 1 ; last 1)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
Time for file removal 42 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8