== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 01:21:34 (1773120094)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.62 seconds: 16.22 ops/second
 - mkdir 77 (time 1773120118.85 total 10.03 last 7.68)
 - mkdir 185 (time 1773120128.91 total 20.09 last 10.74)
 - mkdir 292 (time 1773120138.94 total 30.12 last 10.66)
 - mkdir 402 (time 1773120148.96 total 40.14 last 10.98)
 - mkdir 517 (time 1773120158.97 total 50.15 last 11.49)
 - mkdir 664 (time 1773120169.01 total 60.19 last 14.64)
 - mkdir 748 (time 1773120179.09 total 70.27 last 8.33)
 - mkdir 836 (time 1773120189.14 total 80.32 last 8.76)
 - mkdir 942 (time 1773120199.24 total 90.42 last 10.50)
total: 1000 mkdir in 97.32 seconds: 10.28 ops/second
 - mkdir 103 (time 1773120125.72 total 10.01 last 10.29)
 - mkdir 222 (time 1773120135.82 total 20.11 last 11.78)
 - mkdir 340 (time 1773120145.89 total 30.19 last 11.71)
 - mkdir 454 (time 1773120155.90 total 40.20 last 11.39)
 - mkdir 597 (time 1773120165.95 total 50.24 last 14.23)
 - mkdir 689 (time 1773120176.07 total 60.36 last 9.09)
 - mkdir 786 (time 1773120186.14 total 70.43 last 9.64)
 - mkdir 875 (time 1773120196.14 total 80.43 last 8.90)
 - mkdir 991 (time 1773120206.21 total 90.51 last 11.51)
total: 1000 mkdir in 92.79 seconds: 10.78 ops/second
 - mkdir 96 (time 1773120123.79 total 10.02 last 9.58)
 - mkdir 204 (time 1773120133.86 total 20.09 last 10.72)
 - mkdir 320 (time 1773120143.87 total 30.10 last 11.59)
 - mkdir 436 (time 1773120153.92 total 40.14 last 11.55)
 - mkdir 563 (time 1773120163.92 total 50.15 last 12.69)
 - mkdir 682 (time 1773120174.00 total 60.23 last 11.80)
 - mkdir 772 (time 1773120184.07 total 70.30 last 8.94)
 - mkdir 861 (time 1773120194.09 total 80.32 last 8.89)
 - mkdir 974 (time 1773120204.09 total 90.32 last 11.29)
total: 1000 mkdir in 94.75 seconds: 10.55 ops/second
 - mkdir 97 (time 1773120124.02 total 10.01 last 9.69)
 - mkdir 206 (time 1773120134.03 total 20.02 last 10.89)
 - mkdir 316 (time 1773120144.06 total 30.05 last 10.97)
 - mkdir 434 (time 1773120154.07 total 40.06 last 11.79)
 - mkdir 567 (time 1773120164.10 total 50.09 last 13.26)
 - mkdir 680 (time 1773120174.13 total 60.12 last 11.26)
 - mkdir 773 (time 1773120184.24 total 70.23 last 9.20)
 - mkdir 860 (time 1773120194.25 total 80.24 last 8.69)
 - mkdir 968 (time 1773120204.33 total 90.32 last 10.71)
total: 1000 mkdir in 96.31 seconds: 10.38 ops/second
 - mkdir 102 (time 1773120125.26 total 10.10 last 10.10)
 - mkdir 214 (time 1773120135.33 total 20.16 last 11.13)
 - mkdir 330 (time 1773120145.46 total 30.29 last 11.45)
 - mkdir 447 (time 1773120155.50 total 40.33 last 11.65)
 - mkdir 589 (time 1773120165.55 total 50.39 last 14.13)
 - mkdir 691 (time 1773120175.66 total 60.50 last 10.09)
 - mkdir 781 (time 1773120185.72 total 70.55 last 8.95)
 - mkdir 869 (time 1773120195.84 total 80.67 last 8.70)
 - mkdir 978 (time 1773120205.86 total 90.70 last 10.87)
total: 1000 mkdir in 95.96 seconds: 10.42 ops/second
 - mkdir 103 (time 1773120125.91 total 10.05 last 10.25)
 - mkdir 210 (time 1773120135.91 total 20.06 last 10.69)
 - mkdir 323 (time 1773120146.00 total 30.14 last 11.21)
 - mkdir 441 (time 1773120156.01 total 40.16 last 11.78)
 - mkdir 587 (time 1773120166.04 total 50.19 last 14.56)
 - mkdir 682 (time 1773120176.16 total 60.30 last 9.39)
 - mkdir 781 (time 1773120186.23 total 70.37 last 9.83)
 - mkdir 866 (time 1773120196.23 total 80.37 last 8.50)
 - mkdir 980 (time 1773120206.31 total 90.46 last 11.30)
total: 1000 mkdir in 95.65 seconds: 10.46 ops/second
 - mkdir 102 (time 1773120125.43 total 10.07 last 10.13)
 - mkdir 209 (time 1773120135.45 total 20.09 last 10.68)
 - mkdir 323 (time 1773120145.47 total 30.12 last 11.37)
 - mkdir 437 (time 1773120155.47 total 40.12 last 11.40)
 - mkdir 575 (time 1773120165.53 total 50.18 last 13.72)
 - mkdir 677 (time 1773120175.66 total 60.30 last 10.07)
 - mkdir 773 (time 1773120185.72 total 70.36 last 9.55)
 - mkdir 862 (time 1773120195.77 total 80.41 last 8.85)
 - mkdir 974 (time 1773120205.91 total 90.56 last 11.04)
total: 1000 mkdir in 96.93 seconds: 10.32 ops/second
 - mkdir 109 (time 1773120126.11 total 10.03 last 10.86)
 - mkdir 224 (time 1773120136.13 total 20.05 last 11.48)
 - mkdir 334 (time 1773120146.16 total 30.08 last 10.97)
 - mkdir 447 (time 1773120156.17 total 40.09 last 11.29)
 - mkdir 587 (time 1773120166.17 total 50.09 last 14.00)
 - mkdir 681 (time 1773120176.17 total 60.10 last 9.40)
 - mkdir 778 (time 1773120186.19 total 70.12 last 9.68)
 - mkdir 864 (time 1773120196.24 total 80.16 last 8.56)
 - mkdir 974 (time 1773120206.36 total 90.28 last 10.87)
total: 1000 mkdir in 96.62 seconds: 10.35 ops/second
 - mkdir 95 (time 1773120125.35 total 10.04 last 9.46)
 - mkdir 206 (time 1773120135.36 total 20.05 last 11.09)
 - mkdir 315 (time 1773120145.36 total 30.06 last 10.90)
 - mkdir 429 (time 1773120155.46 total 40.16 last 11.29)
 - mkdir 566 (time 1773120165.48 total 50.18 last 13.67)
 - mkdir 669 (time 1773120175.48 total 60.18 last 10.30)
 - mkdir 762 (time 1773120185.50 total 70.20 last 9.28)
 - mkdir 850 (time 1773120195.53 total 80.23 last 8.77)
 - mkdir 960 (time 1773120205.54 total 90.24 last 10.99)
total: 1000 mkdir in 100.08 seconds: 9.99 ops/second
 - mkdir 106 (time 1773120126.41 total 10.06 last 10.54)
 - mkdir 211 (time 1773120136.44 total 20.09 last 10.47)
 - mkdir 323 (time 1773120146.54 total 30.19 last 11.08)
 - mkdir 442 (time 1773120156.60 total 40.25 last 11.84)
 - mkdir 590 (time 1773120166.63 total 50.28 last 14.75)
 - mkdir 677 (time 1773120176.65 total 60.30 last 8.69)
 - mkdir 771 (time 1773120186.72 total 70.37 last 9.33)
 - mkdir 858 (time 1773120196.73 total 80.38 last 8.69)
 - mkdir 958 (time 1773120206.83 total 90.48 last 9.90)
total: 1000 mkdir in 99.85 seconds: 10.01 ops/second
Time for file creation 122 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 25 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.92 seconds: 10.87 ops/second
 - mkdir 85 (time 1773121118.14 total 10.18 last 8.35)
 - mkdir 185 (time 1773121128.18 total 20.22 last 9.96)
 - mkdir 279 (time 1773121138.29 total 30.33 last 9.29)
 - mkdir 377 (time 1773121148.33 total 40.37 last 9.76)
 - mkdir 471 (time 1773121158.33 total 50.38 last 9.40)
 - mkdir 580 (time 1773121168.34 total 60.38 last 10.89)
 - mkdir 668 (time 1773121178.37 total 70.41 last 8.78)
 - mkdir 763 (time 1773121188.42 total 80.46 last 9.45)
 - mkdir 849 (time 1773121198.50 total 90.55 last 8.53)
 - mkdir 934 (time 1773121208.54 total 100.58 last 8.47)
total: 1000 mkdir in 107.00 seconds: 9.35 ops/second
 - mkdir 94 (time 1773121124.93 total 10.16 last 9.26)
 - mkdir 198 (time 1773121134.95 total 20.18 last 10.38)
 - mkdir 297 (time 1773121145.00 total 30.23 last 9.85)
 - mkdir 396 (time 1773121155.08 total 40.31 last 9.82)
 - mkdir 493 (time 1773121165.08 total 50.31 last 9.70)
 - mkdir 591 (time 1773121175.16 total 60.39 last 9.72)
 - mkdir 686 (time 1773121185.16 total 70.39 last 9.50)
 - mkdir 785 (time 1773121195.28 total 80.51 last 9.79)
 - mkdir 877 (time 1773121205.36 total 90.59 last 9.13)
 - mkdir 972 (time 1773121215.36 total 100.59 last 9.50)
total: 1000 mkdir in 106.23 seconds: 9.41 ops/second
 - mkdir 96 (time 1773121124.73 total 10.04 last 9.56)
 - mkdir 194 (time 1773121134.81 total 20.13 last 9.72)
 - mkdir 291 (time 1773121144.83 total 30.15 last 9.68)
 - mkdir 392 (time 1773121154.91 total 40.22 last 10.03)
 - mkdir 495 (time 1773121164.94 total 50.26 last 10.26)
 - mkdir 594 (time 1773121174.95 total 60.27 last 9.89)
 - mkdir 685 (time 1773121185.03 total 70.35 last 9.03)
 - mkdir 782 (time 1773121195.06 total 80.38 last 9.67)
 - mkdir 872 (time 1773121205.09 total 90.41 last 8.97)
 - mkdir 966 (time 1773121215.23 total 100.55 last 9.27)
total: 1000 mkdir in 108.87 seconds: 9.19 ops/second
 - mkdir 73 (time 1773121119.92 total 10.01 last 7.29)
 - mkdir 169 (time 1773121129.94 total 20.03 last 9.58)
 - mkdir 262 (time 1773121139.97 total 30.07 last 9.26)
 - mkdir 360 (time 1773121150.00 total 40.09 last 9.78)
 - mkdir 453 (time 1773121160.05 total 50.14 last 9.25)
 - mkdir 558 (time 1773121170.05 total 60.15 last 10.50)
 - mkdir 644 (time 1773121180.10 total 70.20 last 8.56)
 - mkdir 736 (time 1773121190.20 total 80.29 last 9.11)
 - mkdir 826 (time 1773121200.29 total 90.39 last 8.92)
 - mkdir 911 (time 1773121210.33 total 100.42 last 8.47)
 - mkdir 983 (time 1773121220.37 total 110.46 last 7.17)
total: 1000 mkdir in 115.75 seconds: 8.64 ops/second
 - mkdir 85 (time 1773121123.15 total 10.01 last 8.49)
 - mkdir 183 (time 1773121133.18 total 20.04 last 9.78)
 - mkdir 281 (time 1773121143.23 total 30.09 last 9.75)
 - mkdir 377 (time 1773121153.31 total 40.17 last 9.52)
 - mkdir 473 (time 1773121163.32 total 50.18 last 9.60)
 - mkdir 575 (time 1773121173.32 total 60.18 last 10.20)
 - mkdir 665 (time 1773121183.33 total 70.19 last 8.99)
 - mkdir 763 (time 1773121193.38 total 80.24 last 9.75)
 - mkdir 850 (time 1773121203.52 total 90.38 last 8.58)
 - mkdir 942 (time 1773121213.56 total 100.42 last 9.16)
 - mkdir 992 (time 1773121223.69 total 110.56 last 4.93)
total: 1000 mkdir in 113.19 seconds: 8.83 ops/second
 - mkdir 70 (time 1773121119.00 total 10.10 last 6.93)
 - mkdir 162 (time 1773121129.05 total 20.15 last 9.16)
 - mkdir 251 (time 1773121139.07 total 30.17 last 8.88)
 - mkdir 345 (time 1773121149.10 total 40.19 last 9.38)
 - mkdir 443 (time 1773121159.19 total 50.29 last 9.71)
 - mkdir 548 (time 1773121169.22 total 60.32 last 10.47)
 - mkdir 632 (time 1773121179.25 total 70.34 last 8.38)
 - mkdir 722 (time 1773121189.28 total 80.38 last 8.97)
 - mkdir 809 (time 1773121199.32 total 90.42 last 8.66)
 - mkdir 893 (time 1773121209.34 total 100.44 last 8.38)
 - mkdir 968 (time 1773121219.50 total 110.59 last 7.39)
total: 1000 mkdir in 118.90 seconds: 8.41 ops/second
 - mkdir 88 (time 1773121124.64 total 10.08 last 8.73)
 - mkdir 188 (time 1773121134.65 total 20.09 last 9.99)
 - mkdir 284 (time 1773121144.70 total 30.14 last 9.56)
 - mkdir 378 (time 1773121154.73 total 40.18 last 9.36)
 - mkdir 479 (time 1773121164.76 total 50.20 last 10.08)
 - mkdir 578 (time 1773121174.85 total 60.29 last 9.81)
 - mkdir 668 (time 1773121184.88 total 70.32 last 8.97)
 - mkdir 764 (time 1773121194.94 total 80.38 last 9.54)
 - mkdir 852 (time 1773121204.99 total 90.43 last 8.76)
 - mkdir 947 (time 1773121215.13 total 100.57 last 9.36)
 - mkdir 984 (time 1773121225.25 total 110.69 last 3.66)
total: 1000 mkdir in 114.66 seconds: 8.72 ops/second
 - mkdir 90 (time 1773121123.43 total 10.05 last 8.96)
 - mkdir 180 (time 1773121133.57 total 20.19 last 8.87)
 - mkdir 277 (time 1773121143.69 total 30.31 last 9.59)
 - mkdir 377 (time 1773121153.73 total 40.35 last 9.96)
 - mkdir 474 (time 1773121163.75 total 50.37 last 9.68)
 - mkdir 571 (time 1773121173.84 total 60.46 last 9.61)
 - mkdir 661 (time 1773121183.90 total 70.52 last 8.94)
 - mkdir 763 (time 1773121193.97 total 80.59 last 10.14)
 - mkdir 848 (time 1773121204.12 total 90.74 last 8.37)
 - mkdir 937 (time 1773121214.22 total 100.84 last 8.81)
 - mkdir 982 (time 1773121224.69 total 111.31 last 4.30)
total: 1000 mkdir in 116.30 seconds: 8.60 ops/second
 - mkdir 81 (time 1773121123.48 total 10.01 last 8.09)
 - mkdir 178 (time 1773121133.53 total 20.06 last 9.66)
 - mkdir 278 (time 1773121143.59 total 30.12 last 9.94)
 - mkdir 378 (time 1773121153.69 total 40.22 last 9.90)
 - mkdir 477 (time 1773121163.75 total 50.28 last 9.85)
 - mkdir 578 (time 1773121173.87 total 60.40 last 9.97)
 - mkdir 662 (time 1773121183.97 total 70.49 last 8.32)
 - mkdir 757 (time 1773121193.97 total 80.50 last 9.50)
 - mkdir 847 (time 1773121204.12 total 90.65 last 8.86)
 - mkdir 936 (time 1773121214.16 total 100.69 last 8.87)
 - mkdir 980 (time 1773121224.52 total 111.05 last 4.25)
total: 1000 mkdir in 116.21 seconds: 8.61 ops/second
 - mkdir 87 (time 1773121124.69 total 10.03 last 8.67)
 - mkdir 183 (time 1773121134.85 total 20.19 last 9.45)
 - mkdir 276 (time 1773121144.86 total 30.20 last 9.29)
 - mkdir 376 (time 1773121154.92 total 40.26 last 9.94)
 - mkdir 477 (time 1773121165.04 total 50.37 last 9.98)
 - mkdir 577 (time 1773121175.12 total 60.46 last 9.92)
 - mkdir 668 (time 1773121185.14 total 70.48 last 9.08)
 - mkdir 762 (time 1773121195.27 total 80.60 last 9.29)
 - mkdir 850 (time 1773121205.35 total 90.69 last 8.73)
 - mkdir 944 (time 1773121215.59 total 100.92 last 9.18)
 - mkdir 984 (time 1773121225.75 total 111.08 last 3.94)
total: 1000 mkdir in 115.11 seconds: 8.69 ops/second
Time for file creation 138 sec for 1 RPC sent at a time
 - unlinked 0 (time 1773121250 ; total 0 ; last 0)
total: 1000 unlinks in 54 seconds: 18.518518 unlinks/second
 - unlinked 0 (time 1773121252 ; total 0 ; last 0)
total: 1000 unlinks in 53 seconds: 18.867924 unlinks/second
 - unlinked 0 (time 1773121252 ; total 0 ; last 0)
total: 1000 unlinks in 55 seconds: 18.181818 unlinks/second
 - unlinked 0 (time 1773121252 ; total 0 ; last 0)
total: 1000 unlinks in 55 seconds: 18.181818 unlinks/second
 - unlinked 0 (time 1773121253 ; total 1 ; last 1)
total: 1000 unlinks in 55 seconds: 18.181818 unlinks/second
 - unlinked 0 (time 1773121254 ; total 1 ; last 1)
total: 1000 unlinks in 54 seconds: 18.518518 unlinks/second
 - unlinked 0 (time 1773121253 ; total 0 ; last 0)
total: 1000 unlinks in 55 seconds: 18.181818 unlinks/second
 - unlinked 0 (time 1773121253 ; total 0 ; last 0)
total: 1000 unlinks in 55 seconds: 18.181818 unlinks/second
 - unlinked 0 (time 1773121253 ; total 0 ; last 0)
total: 1000 unlinks in 55 seconds: 18.181818 unlinks/second
 - unlinked 0 (time 1773121254 ; total 1 ; last 1)
total: 1000 unlinks in 55 seconds: 18.181818 unlinks/second
Time for file removal 81 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8