== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 17:01:07 (1787605267)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.63 seconds: 15.87 ops/second
 - mkdir 141 (time 1787605291.04 total 10.13 last 13.92)
 - mkdir 302 (time 1787605301.07 total 20.16 last 16.05)
 - mkdir 451 (time 1787605311.10 total 30.19 last 14.86)
 - mkdir 611 (time 1787605321.15 total 40.25 last 15.91)
 - mkdir 753 (time 1787605331.17 total 50.26 last 14.18)
 - mkdir 899 (time 1787605341.20 total 60.29 last 14.55)
total: 1000 mkdir in 65.86 seconds: 15.18 ops/second
 - mkdir 125 (time 1787605289.79 total 10.00 last 12.50)
 - mkdir 278 (time 1787605299.80 total 20.02 last 15.28)
 - mkdir 430 (time 1787605309.86 total 30.07 last 15.12)
 - mkdir 587 (time 1787605319.95 total 40.16 last 15.55)
 - mkdir 724 (time 1787605329.97 total 50.18 last 13.68)
 - mkdir 876 (time 1787605340.02 total 60.23 last 15.12)
total: 1000 mkdir in 68.14 seconds: 14.67 ops/second
 - mkdir 141 (time 1787605292.54 total 10.06 last 14.02)
 - mkdir 310 (time 1787605302.58 total 20.10 last 16.84)
 - mkdir 462 (time 1787605312.59 total 30.11 last 15.18)
 - mkdir 625 (time 1787605322.61 total 40.12 last 16.28)
 - mkdir 774 (time 1787605332.64 total 50.16 last 14.85)
 - mkdir 912 (time 1787605342.75 total 60.27 last 13.65)
total: 1000 mkdir in 65.49 seconds: 15.27 ops/second
 - mkdir 129 (time 1787605292.59 total 10.14 last 12.73)
 - mkdir 292 (time 1787605302.67 total 20.22 last 16.16)
 - mkdir 441 (time 1787605312.73 total 30.28 last 14.82)
 - mkdir 603 (time 1787605322.81 total 40.36 last 16.07)
 - mkdir 761 (time 1787605332.84 total 50.39 last 15.75)
 - mkdir 899 (time 1787605342.92 total 60.47 last 13.70)
total: 1000 mkdir in 67.13 seconds: 14.90 ops/second
 - mkdir 131 (time 1787605291.83 total 10.02 last 13.07)
 - mkdir 288 (time 1787605301.89 total 20.08 last 15.61)
 - mkdir 442 (time 1787605311.96 total 30.14 last 15.30)
 - mkdir 603 (time 1787605321.99 total 40.18 last 16.05)
 - mkdir 748 (time 1787605332.05 total 50.23 last 14.42)
 - mkdir 889 (time 1787605342.08 total 60.27 last 14.05)
total: 1000 mkdir in 67.79 seconds: 14.75 ops/second
 - mkdir 130 (time 1787605291.53 total 10.01 last 12.98)
 - mkdir 292 (time 1787605301.56 total 20.04 last 16.16)
 - mkdir 437 (time 1787605311.59 total 30.07 last 14.45)
 - mkdir 595 (time 1787605321.64 total 40.12 last 15.72)
 - mkdir 742 (time 1787605331.65 total 50.14 last 14.68)
 - mkdir 882 (time 1787605341.70 total 60.18 last 13.93)
total: 1000 mkdir in 68.17 seconds: 14.67 ops/second
 - mkdir 131 (time 1787605291.81 total 10.01 last 13.09)
 - mkdir 288 (time 1787605301.84 total 20.04 last 15.65)
 - mkdir 433 (time 1787605311.85 total 30.05 last 14.49)
 - mkdir 589 (time 1787605321.90 total 40.10 last 15.52)
 - mkdir 737 (time 1787605332.00 total 50.20 last 14.66)
 - mkdir 884 (time 1787605342.03 total 60.23 last 14.66)
total: 1000 mkdir in 68.41 seconds: 14.62 ops/second
 - mkdir 132 (time 1787605291.89 total 10.15 last 13.01)
 - mkdir 291 (time 1787605301.93 total 20.19 last 15.83)
 - mkdir 440 (time 1787605311.95 total 30.21 last 14.87)
 - mkdir 594 (time 1787605321.97 total 40.23 last 15.37)
 - mkdir 739 (time 1787605332.00 total 50.26 last 14.46)
 - mkdir 880 (time 1787605342.01 total 60.27 last 14.08)
total: 1000 mkdir in 69.27 seconds: 14.44 ops/second
 - mkdir 127 (time 1787605292.38 total 10.18 last 12.47)
 - mkdir 288 (time 1787605302.42 total 20.22 last 16.04)
 - mkdir 433 (time 1787605312.42 total 30.22 last 14.50)
 - mkdir 592 (time 1787605322.44 total 40.24 last 15.88)
 - mkdir 741 (time 1787605332.46 total 50.25 last 14.88)
 - mkdir 878 (time 1787605342.51 total 60.30 last 13.63)
total: 1000 mkdir in 69.47 seconds: 14.39 ops/second
 - mkdir 127 (time 1787605292.35 total 10.11 last 12.56)
 - mkdir 284 (time 1787605302.39 total 20.15 last 15.64)
 - mkdir 426 (time 1787605312.53 total 30.29 last 14.00)
 - mkdir 583 (time 1787605322.56 total 40.32 last 15.65)
 - mkdir 732 (time 1787605332.57 total 50.33 last 14.88)
 - mkdir 865 (time 1787605342.59 total 60.36 last 13.27)
 - mkdir 993 (time 1787605352.64 total 70.41 last 12.74)
total: 1000 mkdir in 70.95 seconds: 14.09 ops/second
Time for file creation 83 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.71 seconds: 14.17 ops/second
 - mkdir 137 (time 1787605927.98 total 10.07 last 13.61)
 - mkdir 291 (time 1787605937.99 total 20.08 last 15.39)
 - mkdir 418 (time 1787605948.08 total 30.17 last 12.58)
 - mkdir 533 (time 1787605958.10 total 40.19 last 11.48)
 - mkdir 666 (time 1787605968.17 total 50.26 last 13.21)
 - mkdir 806 (time 1787605978.22 total 60.31 last 13.93)
 - mkdir 944 (time 1787605988.24 total 70.33 last 13.77)
total: 1000 mkdir in 75.07 seconds: 13.32 ops/second
 - mkdir 139 (time 1787605927.81 total 10.06 last 13.82)
 - mkdir 291 (time 1787605937.82 total 20.07 last 15.18)
 - mkdir 412 (time 1787605947.83 total 30.09 last 12.08)
 - mkdir 526 (time 1787605957.89 total 40.14 last 11.34)
 - mkdir 653 (time 1787605967.91 total 50.16 last 12.68)
 - mkdir 795 (time 1787605977.93 total 60.18 last 14.17)
 - mkdir 932 (time 1787605987.97 total 70.22 last 13.65)
total: 1000 mkdir in 77.15 seconds: 12.96 ops/second
 - mkdir 136 (time 1787605927.15 total 10.02 last 13.57)
 - mkdir 281 (time 1787605937.19 total 20.07 last 14.43)
 - mkdir 410 (time 1787605947.36 total 30.23 last 12.70)
 - mkdir 522 (time 1787605957.40 total 40.27 last 11.15)
 - mkdir 646 (time 1787605967.45 total 50.32 last 12.34)
 - mkdir 789 (time 1787605977.53 total 60.41 last 14.18)
 - mkdir 923 (time 1787605987.70 total 70.58 last 13.17)
total: 1000 mkdir in 78.08 seconds: 12.81 ops/second
 - mkdir 139 (time 1787605928.68 total 10.03 last 13.86)
 - mkdir 291 (time 1787605938.70 total 20.04 last 15.18)
 - mkdir 413 (time 1787605948.72 total 30.06 last 12.17)
 - mkdir 520 (time 1787605958.77 total 40.11 last 10.65)
 - mkdir 651 (time 1787605968.82 total 50.17 last 13.02)
 - mkdir 790 (time 1787605978.90 total 60.24 last 13.80)
 - mkdir 926 (time 1787605988.99 total 70.34 last 13.47)
total: 1000 mkdir in 79.39 seconds: 12.60 ops/second
 - mkdir 141 (time 1787605928.87 total 10.08 last 13.99)
 - mkdir 284 (time 1787605938.88 total 20.08 last 14.29)
 - mkdir 410 (time 1787605948.90 total 30.11 last 12.57)
 - mkdir 514 (time 1787605958.90 total 40.11 last 10.40)
 - mkdir 648 (time 1787605968.94 total 50.15 last 13.35)
 - mkdir 785 (time 1787605978.96 total 60.16 last 13.68)
 - mkdir 917 (time 1787605988.97 total 70.17 last 13.19)
 - mkdir 996 (time 1787605999.04 total 80.25 last 7.84)
total: 1000 mkdir in 80.95 seconds: 12.35 ops/second
 - mkdir 134 (time 1787605928.35 total 10.04 last 13.34)
 - mkdir 281 (time 1787605938.39 total 20.09 last 14.64)
 - mkdir 403 (time 1787605948.44 total 30.13 last 12.14)
 - mkdir 514 (time 1787605958.47 total 40.17 last 11.06)
 - mkdir 645 (time 1787605968.47 total 50.17 last 13.10)
 - mkdir 787 (time 1787605978.52 total 60.21 last 14.14)
 - mkdir 915 (time 1787605988.59 total 70.29 last 12.70)
 - mkdir 988 (time 1787605998.77 total 80.47 last 7.17)
total: 1000 mkdir in 82.61 seconds: 12.11 ops/second
 - mkdir 128 (time 1787605928.00 total 10.12 last 12.65)
 - mkdir 272 (time 1787605938.04 total 20.16 last 14.35)
 - mkdir 389 (time 1787605948.15 total 30.27 last 11.57)
 - mkdir 501 (time 1787605958.29 total 40.41 last 11.05)
 - mkdir 628 (time 1787605968.30 total 50.41 last 12.69)
 - mkdir 769 (time 1787605978.37 total 60.49 last 14.00)
 - mkdir 901 (time 1787605988.39 total 70.51 last 13.18)
 - mkdir 985 (time 1787605998.41 total 80.53 last 8.38)
total: 1000 mkdir in 83.10 seconds: 12.03 ops/second
 - mkdir 140 (time 1787605929.12 total 10.07 last 13.90)
 - mkdir 285 (time 1787605939.15 total 20.10 last 14.46)
 - mkdir 410 (time 1787605949.22 total 30.17 last 12.42)
 - mkdir 512 (time 1787605959.30 total 40.25 last 10.12)
 - mkdir 642 (time 1787605969.34 total 50.29 last 12.95)
 - mkdir 776 (time 1787605979.40 total 60.35 last 13.32)
 - mkdir 911 (time 1787605989.46 total 70.41 last 13.41)
 - mkdir 986 (time 1787605999.46 total 80.41 last 7.50)
total: 1000 mkdir in 82.46 seconds: 12.13 ops/second
 - mkdir 131 (time 1787605928.98 total 10.05 last 13.03)
 - mkdir 274 (time 1787605939.02 total 20.09 last 14.24)
 - mkdir 396 (time 1787605949.09 total 30.17 last 12.11)
 - mkdir 499 (time 1787605959.11 total 40.18 last 10.28)
 - mkdir 633 (time 1787605969.17 total 50.25 last 13.32)
 - mkdir 771 (time 1787605979.19 total 60.27 last 13.77)
 - mkdir 898 (time 1787605989.20 total 70.27 last 12.69)
 - mkdir 972 (time 1787605999.21 total 80.29 last 7.39)
total: 1000 mkdir in 83.88 seconds: 11.92 ops/second
 - mkdir 124 (time 1787605928.02 total 10.03 last 12.36)
 - mkdir 262 (time 1787605938.04 total 20.06 last 13.77)
 - mkdir 371 (time 1787605948.08 total 30.10 last 10.86)
 - mkdir 484 (time 1787605958.13 total 40.15 last 11.24)
 - mkdir 612 (time 1787605968.18 total 50.20 last 12.73)
 - mkdir 753 (time 1787605978.29 total 60.31 last 13.95)
 - mkdir 877 (time 1787605988.37 total 70.39 last 12.31)
 - mkdir 956 (time 1787605998.49 total 80.51 last 7.81)
total: 1000 mkdir in 86.42 seconds: 11.57 ops/second
Time for file creation 97 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787606010 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1787606012 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1787606013 ; total 0 ; last 0)
total: 1000 unlinks in 41 seconds: 24.390244 unlinks/second
 - unlinked 0 (time 1787606012 ; total 0 ; last 0)
total: 1000 unlinks in 42 seconds: 23.809525 unlinks/second
 - unlinked 0 (time 1787606013 ; total 0 ; last 0)
total: 1000 unlinks in 42 seconds: 23.809525 unlinks/second
 - unlinked 0 (time 1787606013 ; total 0 ; last 0)
total: 1000 unlinks in 42 seconds: 23.809525 unlinks/second
 - unlinked 0 (time 1787606013 ; total 0 ; last 0)
total: 1000 unlinks in 43 seconds: 23.255814 unlinks/second
 - unlinked 0 (time 1787606013 ; total 0 ; last 0)
total: 1000 unlinks in 43 seconds: 23.255814 unlinks/second
 - unlinked 0 (time 1787606013 ; total 0 ; last 0)
total: 1000 unlinks in 43 seconds: 23.255814 unlinks/second
 - unlinked 0 (time 1787606013 ; total 0 ; last 0)
total: 1000 unlinks in 43 seconds: 23.255814 unlinks/second
Time for file removal 55 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8