== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 15:52:30 (1787601150)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.56 seconds: 17.74 ops/second
 - mkdir 137 (time 1787601174.01 total 10.01 last 13.69)
 - mkdir 299 (time 1787601184.05 total 20.05 last 16.13)
 - mkdir 463 (time 1787601194.05 total 30.05 last 16.40)
 - mkdir 652 (time 1787601204.09 total 40.09 last 18.83)
 - mkdir 834 (time 1787601214.13 total 50.13 last 18.12)
total: 1000 mkdir in 58.80 seconds: 17.01 ops/second
 - mkdir 139 (time 1787601174.05 total 10.06 last 13.82)
 - mkdir 300 (time 1787601184.09 total 20.10 last 16.04)
 - mkdir 466 (time 1787601194.16 total 30.16 last 16.49)
 - mkdir 651 (time 1787601204.17 total 40.18 last 18.47)
 - mkdir 825 (time 1787601214.21 total 50.22 last 17.34)
total: 1000 mkdir in 59.53 seconds: 16.80 ops/second
 - mkdir 131 (time 1787601173.10 total 10.05 last 13.04)
 - mkdir 283 (time 1787601183.12 total 20.07 last 15.16)
 - mkdir 445 (time 1787601193.15 total 30.10 last 16.15)
 - mkdir 630 (time 1787601203.18 total 40.13 last 18.45)
 - mkdir 811 (time 1787601213.22 total 50.17 last 18.02)
 - mkdir 993 (time 1787601223.31 total 60.26 last 18.04)
total: 1000 mkdir in 60.74 seconds: 16.46 ops/second
 - mkdir 140 (time 1787601173.53 total 10.02 last 13.98)
 - mkdir 299 (time 1787601183.59 total 20.07 last 15.81)
 - mkdir 460 (time 1787601193.61 total 30.09 last 16.08)
 - mkdir 642 (time 1787601203.62 total 40.10 last 18.18)
 - mkdir 814 (time 1787601213.63 total 50.11 last 17.19)
 - mkdir 994 (time 1787601223.63 total 60.12 last 17.98)
total: 1000 mkdir in 60.50 seconds: 16.53 ops/second
 - mkdir 138 (time 1787601173.92 total 10.05 last 13.74)
 - mkdir 294 (time 1787601183.95 total 20.08 last 15.54)
 - mkdir 460 (time 1787601193.98 total 30.11 last 16.56)
 - mkdir 641 (time 1787601203.99 total 40.12 last 18.07)
 - mkdir 818 (time 1787601214.06 total 50.18 last 17.59)
 - mkdir 998 (time 1787601224.17 total 60.30 last 17.80)
total: 1000 mkdir in 60.38 seconds: 16.56 ops/second
 - mkdir 135 (time 1787601173.86 total 10.00 last 13.50)
 - mkdir 289 (time 1787601183.90 total 20.04 last 15.34)
 - mkdir 449 (time 1787601193.96 total 30.10 last 15.91)
 - mkdir 635 (time 1787601203.97 total 40.11 last 18.57)
 - mkdir 817 (time 1787601213.99 total 50.13 last 18.17)
 - mkdir 992 (time 1787601223.99 total 60.13 last 17.50)
total: 1000 mkdir in 60.64 seconds: 16.49 ops/second
 - mkdir 135 (time 1787601172.62 total 10.04 last 13.45)
 - mkdir 284 (time 1787601182.67 total 20.09 last 14.83)
 - mkdir 443 (time 1787601192.70 total 30.11 last 15.86)
 - mkdir 614 (time 1787601202.74 total 40.16 last 17.03)
 - mkdir 790 (time 1787601212.75 total 50.17 last 17.58)
 - mkdir 972 (time 1787601222.77 total 60.18 last 18.17)
total: 1000 mkdir in 62.01 seconds: 16.13 ops/second
 - mkdir 131 (time 1787601173.94 total 10.02 last 13.07)
 - mkdir 286 (time 1787601183.98 total 20.06 last 15.44)
 - mkdir 448 (time 1787601194.00 total 30.09 last 16.16)
 - mkdir 629 (time 1787601204.01 total 40.09 last 18.09)
 - mkdir 808 (time 1787601214.06 total 50.14 last 17.81)
 - mkdir 988 (time 1787601224.16 total 60.24 last 17.82)
total: 1000 mkdir in 60.90 seconds: 16.42 ops/second
 - mkdir 133 (time 1787601174.20 total 10.01 last 13.28)
 - mkdir 290 (time 1787601184.22 total 20.03 last 15.67)
 - mkdir 450 (time 1787601194.24 total 30.05 last 15.97)
 - mkdir 628 (time 1787601204.27 total 40.08 last 17.75)
 - mkdir 801 (time 1787601214.34 total 50.16 last 17.17)
 - mkdir 982 (time 1787601224.36 total 60.17 last 18.08)
total: 1000 mkdir in 61.49 seconds: 16.26 ops/second
 - mkdir 131 (time 1787601174.37 total 10.03 last 13.06)
 - mkdir 282 (time 1787601184.41 total 20.07 last 15.04)
 - mkdir 438 (time 1787601194.49 total 30.15 last 15.47)
 - mkdir 609 (time 1787601204.53 total 40.20 last 17.03)
 - mkdir 776 (time 1787601214.57 total 50.23 last 16.64)
 - mkdir 943 (time 1787601224.59 total 60.25 last 16.66)
total: 1000 mkdir in 63.67 seconds: 15.71 ops/second
Time for file creation 74 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.39 seconds: 25.69 ops/second
 - mkdir 161 (time 1787601768.73 total 10.05 last 16.02)
 - mkdir 316 (time 1787601778.75 total 20.07 last 15.47)
 - mkdir 467 (time 1787601788.81 total 30.13 last 15.02)
 - mkdir 596 (time 1787601798.83 total 40.15 last 12.87)
 - mkdir 718 (time 1787601808.85 total 50.17 last 12.17)
 - mkdir 852 (time 1787601818.88 total 60.20 last 13.36)
total: 1000 mkdir in 69.88 seconds: 14.31 ops/second
 - mkdir 152 (time 1787601771.40 total 10.01 last 15.19)
 - mkdir 300 (time 1787601781.40 total 20.01 last 14.80)
 - mkdir 459 (time 1787601791.40 total 30.01 last 15.89)
 - mkdir 580 (time 1787601801.44 total 40.05 last 12.06)
 - mkdir 706 (time 1787601811.55 total 50.15 last 12.47)
 - mkdir 852 (time 1787601821.55 total 60.16 last 14.59)
 - mkdir 982 (time 1787601831.68 total 70.29 last 12.84)
total: 1000 mkdir in 72.78 seconds: 13.74 ops/second
 - mkdir 154 (time 1787601770.50 total 10.01 last 15.39)
 - mkdir 298 (time 1787601780.50 total 20.01 last 14.39)
 - mkdir 452 (time 1787601790.62 total 30.13 last 15.22)
 - mkdir 577 (time 1787601800.64 total 40.15 last 12.47)
 - mkdir 702 (time 1787601810.66 total 50.17 last 12.48)
 - mkdir 841 (time 1787601820.71 total 60.22 last 13.84)
 - mkdir 974 (time 1787601830.71 total 70.22 last 13.29)
total: 1000 mkdir in 74.26 seconds: 13.47 ops/second
 - mkdir 147 (time 1787601770.29 total 10.00 last 14.70)
 - mkdir 294 (time 1787601780.32 total 20.04 last 14.65)
 - mkdir 446 (time 1787601790.36 total 30.07 last 15.15)
 - mkdir 568 (time 1787601800.37 total 40.08 last 12.19)
 - mkdir 694 (time 1787601810.38 total 50.10 last 12.58)
 - mkdir 835 (time 1787601820.44 total 60.15 last 14.02)
 - mkdir 970 (time 1787601830.55 total 70.27 last 13.35)
total: 1000 mkdir in 74.78 seconds: 13.37 ops/second
 - mkdir 152 (time 1787601771.34 total 10.02 last 15.17)
 - mkdir 300 (time 1787601781.35 total 20.03 last 14.79)
 - mkdir 453 (time 1787601791.37 total 30.04 last 15.28)
 - mkdir 574 (time 1787601801.37 total 40.05 last 12.09)
 - mkdir 699 (time 1787601811.37 total 50.05 last 12.49)
 - mkdir 843 (time 1787601821.42 total 60.09 last 14.34)
 - mkdir 971 (time 1787601831.55 total 70.22 last 12.64)
total: 1000 mkdir in 74.90 seconds: 13.35 ops/second
 - mkdir 152 (time 1787601771.15 total 10.09 last 15.07)
 - mkdir 301 (time 1787601781.16 total 20.10 last 14.88)
 - mkdir 447 (time 1787601791.16 total 30.10 last 14.60)
 - mkdir 572 (time 1787601801.25 total 40.19 last 12.39)
 - mkdir 697 (time 1787601811.27 total 50.21 last 12.47)
 - mkdir 844 (time 1787601821.30 total 60.24 last 14.66)
 - mkdir 968 (time 1787601831.32 total 70.26 last 12.38)
total: 1000 mkdir in 75.40 seconds: 13.26 ops/second
 - mkdir 150 (time 1787601771.64 total 10.00 last 15.00)
 - mkdir 295 (time 1787601781.68 total 20.05 last 14.43)
 - mkdir 451 (time 1787601791.69 total 30.06 last 15.59)
 - mkdir 573 (time 1787601801.72 total 40.09 last 12.16)
 - mkdir 697 (time 1787601811.77 total 50.14 last 12.34)
 - mkdir 845 (time 1787601821.77 total 60.14 last 14.80)
 - mkdir 972 (time 1787601831.83 total 70.20 last 12.63)
total: 1000 mkdir in 75.30 seconds: 13.28 ops/second
 - mkdir 149 (time 1787601770.81 total 10.05 last 14.82)
 - mkdir 292 (time 1787601780.82 total 20.06 last 14.29)
 - mkdir 440 (time 1787601790.85 total 30.09 last 14.75)
 - mkdir 567 (time 1787601800.89 total 40.14 last 12.64)
 - mkdir 695 (time 1787601810.95 total 50.20 last 12.72)
 - mkdir 836 (time 1787601821.05 total 60.29 last 13.97)
 - mkdir 959 (time 1787601831.17 total 70.41 last 12.15)
total: 1000 mkdir in 76.58 seconds: 13.06 ops/second
 - mkdir 145 (time 1787601771.32 total 10.01 last 14.49)
 - mkdir 292 (time 1787601781.35 total 20.04 last 14.65)
 - mkdir 438 (time 1787601791.38 total 30.07 last 14.55)
 - mkdir 560 (time 1787601801.39 total 40.08 last 12.19)
 - mkdir 689 (time 1787601811.43 total 50.12 last 12.85)
 - mkdir 834 (time 1787601821.47 total 60.16 last 14.45)
 - mkdir 960 (time 1787601831.56 total 70.25 last 12.48)
total: 1000 mkdir in 76.54 seconds: 13.06 ops/second
 - mkdir 138 (time 1787601771.25 total 10.07 last 13.71)
 - mkdir 281 (time 1787601781.29 total 20.10 last 14.25)
 - mkdir 424 (time 1787601791.34 total 30.15 last 14.23)
 - mkdir 547 (time 1787601801.37 total 40.18 last 12.26)
 - mkdir 676 (time 1787601811.46 total 50.27 last 12.78)
 - mkdir 818 (time 1787601821.48 total 60.29 last 14.18)
 - mkdir 938 (time 1787601831.52 total 70.33 last 11.95)
total: 1000 mkdir in 78.48 seconds: 12.74 ops/second
Time for file creation 87 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787601846 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787601847 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787601848 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787601848 ; total 1 ; last 1)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787601848 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787601848 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787601848 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787601849 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787601848 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787601848 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
Time for file removal 44 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8