== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 20:11:47 (1790122307)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.48 seconds: 20.94 ops/second
 - mkdir 150 (time 1790122329.23 total 10.09 last 14.87)
 - mkdir 306 (time 1790122339.25 total 20.11 last 15.56)
 - mkdir 483 (time 1790122349.26 total 30.12 last 17.69)
 - mkdir 647 (time 1790122359.31 total 40.17 last 16.33)
 - mkdir 785 (time 1790122369.35 total 50.21 last 13.74)
 - mkdir 959 (time 1790122379.36 total 60.22 last 17.38)
total: 1000 mkdir in 62.91 seconds: 15.90 ops/second
 - mkdir 156 (time 1790122329.85 total 10.01 last 15.59)
 - mkdir 312 (time 1790122339.85 total 20.02 last 15.59)
 - mkdir 495 (time 1790122349.87 total 30.03 last 18.27)
 - mkdir 652 (time 1790122359.88 total 40.05 last 15.68)
 - mkdir 788 (time 1790122369.89 total 50.05 last 13.59)
 - mkdir 956 (time 1790122379.89 total 60.06 last 16.79)
total: 1000 mkdir in 63.03 seconds: 15.86 ops/second
 - mkdir 167 (time 1790122331.12 total 10.04 last 16.63)
 - mkdir 315 (time 1790122341.19 total 20.12 last 14.69)
 - mkdir 517 (time 1790122351.22 total 30.15 last 20.13)
 - mkdir 664 (time 1790122361.26 total 40.18 last 14.65)
 - mkdir 809 (time 1790122371.32 total 50.25 last 14.41)
 - mkdir 977 (time 1790122381.37 total 60.30 last 16.72)
total: 1000 mkdir in 62.03 seconds: 16.12 ops/second
 - mkdir 161 (time 1790122330.99 total 10.02 last 16.08)
 - mkdir 312 (time 1790122341.03 total 20.05 last 15.05)
 - mkdir 507 (time 1790122351.04 total 30.06 last 19.48)
 - mkdir 658 (time 1790122361.07 total 40.09 last 15.05)
 - mkdir 805 (time 1790122371.07 total 50.09 last 14.70)
 - mkdir 970 (time 1790122381.11 total 60.14 last 16.43)
total: 1000 mkdir in 62.41 seconds: 16.02 ops/second
 - mkdir 163 (time 1790122331.30 total 10.00 last 16.29)
 - mkdir 318 (time 1790122341.33 total 20.03 last 15.45)
 - mkdir 520 (time 1790122351.36 total 30.06 last 20.14)
 - mkdir 661 (time 1790122361.36 total 40.06 last 14.10)
 - mkdir 806 (time 1790122371.39 total 50.09 last 14.46)
 - mkdir 970 (time 1790122381.42 total 60.12 last 16.35)
total: 1000 mkdir in 62.85 seconds: 15.91 ops/second
 - mkdir 162 (time 1790122330.96 total 10.04 last 16.14)
 - mkdir 307 (time 1790122340.96 total 20.04 last 14.49)
 - mkdir 501 (time 1790122351.03 total 30.12 last 19.26)
 - mkdir 647 (time 1790122361.08 total 40.16 last 14.54)
 - mkdir 789 (time 1790122371.12 total 50.20 last 14.14)
 - mkdir 956 (time 1790122381.14 total 60.23 last 16.65)
total: 1000 mkdir in 63.93 seconds: 15.64 ops/second
 - mkdir 158 (time 1790122330.80 total 10.02 last 15.77)
 - mkdir 306 (time 1790122340.81 total 20.03 last 14.79)
 - mkdir 503 (time 1790122350.86 total 30.07 last 19.60)
 - mkdir 649 (time 1790122360.95 total 40.16 last 14.47)
 - mkdir 793 (time 1790122370.99 total 50.20 last 14.34)
 - mkdir 958 (time 1790122381.04 total 60.26 last 16.41)
total: 1000 mkdir in 64.29 seconds: 15.55 ops/second
 - mkdir 157 (time 1790122331.61 total 10.00 last 15.69)
 - mkdir 304 (time 1790122341.64 total 20.03 last 14.66)
 - mkdir 508 (time 1790122351.71 total 30.11 last 20.25)
 - mkdir 655 (time 1790122361.75 total 40.15 last 14.64)
 - mkdir 797 (time 1790122371.79 total 50.18 last 14.15)
 - mkdir 965 (time 1790122381.83 total 60.22 last 16.74)
total: 1000 mkdir in 63.89 seconds: 15.65 ops/second
 - mkdir 150 (time 1790122330.61 total 10.02 last 14.98)
 - mkdir 294 (time 1790122340.68 total 20.08 last 14.31)
 - mkdir 484 (time 1790122350.68 total 30.08 last 19.00)
 - mkdir 634 (time 1790122360.76 total 40.16 last 14.88)
 - mkdir 776 (time 1790122370.79 total 50.19 last 14.16)
 - mkdir 942 (time 1790122380.81 total 60.21 last 16.57)
total: 1000 mkdir in 65.82 seconds: 15.19 ops/second
 - mkdir 149 (time 1790122330.52 total 10.02 last 14.87)
 - mkdir 291 (time 1790122340.57 total 20.07 last 14.13)
 - mkdir 473 (time 1790122350.60 total 30.10 last 18.15)
 - mkdir 623 (time 1790122360.63 total 40.13 last 14.96)
 - mkdir 762 (time 1790122370.66 total 50.16 last 13.86)
 - mkdir 929 (time 1790122380.73 total 60.23 last 16.58)
total: 1000 mkdir in 66.99 seconds: 14.93 ops/second
Time for file creation 77 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 14 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.49 seconds: 20.31 ops/second
 - mkdir 150 (time 1790122928.29 total 10.06 last 14.91)
 - mkdir 279 (time 1790122938.31 total 20.08 last 12.87)
 - mkdir 438 (time 1790122948.37 total 30.13 last 15.81)
 - mkdir 578 (time 1790122958.48 total 40.24 last 13.85)
 - mkdir 736 (time 1790122968.52 total 50.28 last 15.74)
 - mkdir 873 (time 1790122978.56 total 60.32 last 13.65)
total: 1000 mkdir in 70.06 seconds: 14.27 ops/second
 - mkdir 143 (time 1790122927.33 total 10.02 last 14.27)
 - mkdir 279 (time 1790122937.37 total 20.06 last 13.55)
 - mkdir 441 (time 1790122947.39 total 30.09 last 16.16)
 - mkdir 580 (time 1790122957.41 total 40.10 last 13.87)
 - mkdir 729 (time 1790122967.42 total 50.11 last 14.89)
 - mkdir 860 (time 1790122977.45 total 60.14 last 13.06)
 - mkdir 986 (time 1790122987.48 total 70.17 last 12.57)
total: 1000 mkdir in 72.02 seconds: 13.88 ops/second
 - mkdir 160 (time 1790122931.04 total 10.01 last 15.99)
 - mkdir 301 (time 1790122941.05 total 20.02 last 14.08)
 - mkdir 460 (time 1790122951.09 total 30.05 last 15.85)
 - mkdir 601 (time 1790122961.09 total 40.05 last 14.10)
 - mkdir 759 (time 1790122971.14 total 50.10 last 15.72)
 - mkdir 895 (time 1790122981.27 total 60.24 last 13.42)
total: 1000 mkdir in 69.47 seconds: 14.39 ops/second
 - mkdir 158 (time 1790122930.73 total 10.03 last 15.75)
 - mkdir 302 (time 1790122940.77 total 20.07 last 14.35)
 - mkdir 463 (time 1790122950.79 total 30.08 last 16.07)
 - mkdir 602 (time 1790122960.85 total 40.15 last 13.81)
 - mkdir 759 (time 1790122970.86 total 50.16 last 15.69)
 - mkdir 895 (time 1790122980.88 total 60.17 last 13.58)
total: 1000 mkdir in 70.16 seconds: 14.25 ops/second
 - mkdir 156 (time 1790122930.58 total 10.01 last 15.58)
 - mkdir 295 (time 1790122940.60 total 20.03 last 13.88)
 - mkdir 457 (time 1790122950.70 total 30.13 last 16.04)
 - mkdir 589 (time 1790122960.71 total 40.14 last 13.18)
 - mkdir 746 (time 1790122970.73 total 50.16 last 15.68)
 - mkdir 884 (time 1790122980.82 total 60.25 last 13.68)
 - mkdir 989 (time 1790122991.04 total 70.47 last 10.28)
total: 1000 mkdir in 72.10 seconds: 13.87 ops/second
 - mkdir 160 (time 1790122930.46 total 10.04 last 15.94)
 - mkdir 297 (time 1790122940.46 total 20.04 last 13.70)
 - mkdir 453 (time 1790122950.51 total 30.09 last 15.52)
 - mkdir 587 (time 1790122960.56 total 40.14 last 13.34)
 - mkdir 742 (time 1790122970.57 total 50.15 last 15.48)
 - mkdir 881 (time 1790122980.59 total 60.17 last 13.87)
 - mkdir 986 (time 1790122990.77 total 70.35 last 10.32)
total: 1000 mkdir in 72.60 seconds: 13.77 ops/second
 - mkdir 167 (time 1790122931.48 total 10.05 last 16.62)
 - mkdir 310 (time 1790122941.51 total 20.08 last 14.25)
 - mkdir 462 (time 1790122951.56 total 30.13 last 15.13)
 - mkdir 600 (time 1790122961.58 total 40.15 last 13.77)
 - mkdir 747 (time 1790122971.61 total 50.18 last 14.66)
 - mkdir 881 (time 1790122981.62 total 60.18 last 13.39)
 - mkdir 981 (time 1790122991.74 total 70.31 last 9.88)
total: 1000 mkdir in 73.48 seconds: 13.61 ops/second
 - mkdir 158 (time 1790122931.18 total 10.07 last 15.69)
 - mkdir 299 (time 1790122941.19 total 20.08 last 14.09)
 - mkdir 452 (time 1790122951.19 total 30.08 last 15.30)
 - mkdir 589 (time 1790122961.21 total 40.09 last 13.68)
 - mkdir 738 (time 1790122971.25 total 50.13 last 14.85)
 - mkdir 869 (time 1790122981.26 total 60.14 last 13.08)
 - mkdir 976 (time 1790122991.57 total 70.46 last 10.37)
total: 1000 mkdir in 74.48 seconds: 13.43 ops/second
 - mkdir 145 (time 1790122930.13 total 10.13 last 14.31)
 - mkdir 279 (time 1790122940.13 total 20.14 last 13.39)
 - mkdir 427 (time 1790122950.18 total 30.18 last 14.74)
 - mkdir 555 (time 1790122960.23 total 40.24 last 12.73)
 - mkdir 703 (time 1790122970.23 total 50.24 last 14.79)
 - mkdir 831 (time 1790122980.25 total 60.26 last 12.78)
 - mkdir 935 (time 1790122990.36 total 70.37 last 10.29)
total: 1000 mkdir in 79.46 seconds: 12.58 ops/second
 - mkdir 144 (time 1790122929.84 total 10.05 last 14.32)
 - mkdir 273 (time 1790122939.88 total 20.09 last 12.86)
 - mkdir 423 (time 1790122949.89 total 30.10 last 14.99)
 - mkdir 550 (time 1790122959.94 total 40.15 last 12.64)
 - mkdir 694 (time 1790122969.97 total 50.18 last 14.36)
 - mkdir 821 (time 1790122980.00 total 60.21 last 12.65)
 - mkdir 924 (time 1790122990.03 total 70.24 last 10.27)
 - mkdir 996 (time 1790123000.12 total 80.33 last 7.14)
total: 1000 mkdir in 80.63 seconds: 12.40 ops/second
Time for file creation 92 sec for 1 RPC sent at a time
 - unlinked 0 (time 1790123010 ; total 1 ; last 1)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1790123011 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1790123012 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1790123013 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1790123013 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1790123014 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1790123014 ; total 1 ; last 1)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1790123014 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1790123014 ; total 1 ; last 1)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1790123014 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
Time for file removal 49 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8