== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 16:16:32 (1788552992)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.51 seconds: 19.73 ops/second
 - mkdir 176 (time 1788553017.31 total 10.02 last 17.57)
 - mkdir 341 (time 1788553027.36 total 20.07 last 16.42)
 - mkdir 552 (time 1788553037.37 total 30.07 last 21.09)
 - mkdir 705 (time 1788553047.46 total 40.16 last 15.16)
 - mkdir 855 (time 1788553057.48 total 50.18 last 14.98)
 - mkdir 977 (time 1788553067.65 total 60.35 last 12.00)
total: 1000 mkdir in 62.74 seconds: 15.94 ops/second
 - mkdir 165 (time 1788553016.77 total 10.02 last 16.47)
 - mkdir 332 (time 1788553026.79 total 20.04 last 16.67)
 - mkdir 535 (time 1788553036.83 total 30.07 last 20.23)
 - mkdir 689 (time 1788553046.86 total 40.11 last 15.34)
 - mkdir 840 (time 1788553056.95 total 50.19 last 14.97)
 - mkdir 963 (time 1788553066.98 total 60.22 last 12.27)
total: 1000 mkdir in 64.49 seconds: 15.51 ops/second
 - mkdir 171 (time 1788553017.50 total 10.00 last 17.10)
 - mkdir 338 (time 1788553027.60 total 20.10 last 16.53)
 - mkdir 541 (time 1788553037.63 total 30.13 last 20.24)
 - mkdir 694 (time 1788553047.69 total 40.19 last 15.22)
 - mkdir 842 (time 1788553057.71 total 50.21 last 14.77)
 - mkdir 961 (time 1788553067.72 total 60.22 last 11.88)
total: 1000 mkdir in 65.39 seconds: 15.29 ops/second
 - mkdir 175 (time 1788553017.84 total 10.06 last 17.39)
 - mkdir 339 (time 1788553027.90 total 20.12 last 16.30)
 - mkdir 548 (time 1788553037.92 total 30.13 last 20.87)
 - mkdir 690 (time 1788553047.96 total 40.18 last 14.13)
 - mkdir 841 (time 1788553057.98 total 50.20 last 15.07)
 - mkdir 960 (time 1788553068.11 total 60.33 last 11.75)
total: 1000 mkdir in 65.45 seconds: 15.28 ops/second
 - mkdir 179 (time 1788553018.42 total 10.01 last 17.89)
 - mkdir 336 (time 1788553028.44 total 20.03 last 15.66)
 - mkdir 554 (time 1788553038.45 total 30.04 last 21.79)
 - mkdir 686 (time 1788553048.45 total 40.04 last 13.19)
 - mkdir 836 (time 1788553058.54 total 50.13 last 14.87)
 - mkdir 956 (time 1788553068.55 total 60.14 last 11.99)
total: 1000 mkdir in 66.60 seconds: 15.01 ops/second
 - mkdir 155 (time 1788553016.66 total 10.01 last 15.49)
 - mkdir 315 (time 1788553026.69 total 20.03 last 15.96)
 - mkdir 514 (time 1788553036.70 total 30.05 last 19.87)
 - mkdir 668 (time 1788553046.73 total 40.07 last 15.36)
 - mkdir 820 (time 1788553056.74 total 50.08 last 15.19)
 - mkdir 941 (time 1788553066.81 total 60.15 last 12.02)
total: 1000 mkdir in 68.96 seconds: 14.50 ops/second
 - mkdir 176 (time 1788553017.96 total 10.00 last 17.59)
 - mkdir 331 (time 1788553028.01 total 20.05 last 15.43)
 - mkdir 537 (time 1788553038.03 total 30.07 last 20.56)
 - mkdir 681 (time 1788553048.10 total 40.14 last 14.30)
 - mkdir 832 (time 1788553058.24 total 50.29 last 14.88)
 - mkdir 950 (time 1788553068.26 total 60.30 last 11.78)
total: 1000 mkdir in 68.13 seconds: 14.68 ops/second
 - mkdir 185 (time 1788553018.48 total 10.01 last 18.48)
 - mkdir 335 (time 1788553028.48 total 20.01 last 15.00)
 - mkdir 547 (time 1788553038.49 total 30.02 last 21.18)
 - mkdir 683 (time 1788553048.53 total 40.06 last 13.55)
 - mkdir 829 (time 1788553058.54 total 50.07 last 14.59)
 - mkdir 954 (time 1788553068.57 total 60.10 last 12.46)
total: 1000 mkdir in 67.76 seconds: 14.76 ops/second
 - mkdir 159 (time 1788553016.53 total 10.05 last 15.83)
 - mkdir 313 (time 1788553026.53 total 20.05 last 15.40)
 - mkdir 507 (time 1788553036.54 total 30.06 last 19.37)
 - mkdir 663 (time 1788553046.63 total 40.14 last 15.47)
 - mkdir 811 (time 1788553056.68 total 50.20 last 14.72)
 - mkdir 931 (time 1788553066.74 total 60.26 last 11.92)
 - mkdir 995 (time 1788553076.87 total 70.39 last 6.32)
total: 1000 mkdir in 70.94 seconds: 14.10 ops/second
 - mkdir 179 (time 1788553018.50 total 10.04 last 17.83)
 - mkdir 330 (time 1788553028.57 total 20.11 last 14.99)
 - mkdir 545 (time 1788553038.64 total 30.19 last 21.34)
 - mkdir 679 (time 1788553048.70 total 40.25 last 13.32)
 - mkdir 828 (time 1788553058.72 total 50.26 last 14.87)
 - mkdir 947 (time 1788553068.76 total 60.31 last 11.85)
total: 1000 mkdir in 69.21 seconds: 14.45 ops/second
Time for file creation 85 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 17 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.80 seconds: 12.51 ops/second
 - mkdir 122 (time 1788553664.81 total 10.02 last 12.17)
 - mkdir 280 (time 1788553674.87 total 20.08 last 15.71)
 - mkdir 403 (time 1788553684.92 total 30.14 last 12.24)
 - mkdir 544 (time 1788553694.95 total 40.17 last 14.05)
 - mkdir 691 (time 1788553704.96 total 50.17 last 14.70)
 - mkdir 868 (time 1788553714.99 total 60.21 last 17.64)
total: 1000 mkdir in 69.93 seconds: 14.30 ops/second
 - mkdir 120 (time 1788553667.14 total 10.04 last 11.95)
 - mkdir 276 (time 1788553677.23 total 20.13 last 15.46)
 - mkdir 412 (time 1788553687.25 total 30.15 last 13.56)
 - mkdir 551 (time 1788553697.36 total 40.26 last 13.75)
 - mkdir 706 (time 1788553707.40 total 50.30 last 15.44)
 - mkdir 883 (time 1788553717.45 total 60.35 last 17.62)
 - mkdir 995 (time 1788553727.66 total 70.56 last 10.97)
total: 1000 mkdir in 71.10 seconds: 14.06 ops/second
 - mkdir 122 (time 1788553667.00 total 10.06 last 12.13)
 - mkdir 279 (time 1788553677.03 total 20.08 last 15.66)
 - mkdir 410 (time 1788553687.07 total 30.13 last 13.04)
 - mkdir 548 (time 1788553697.16 total 40.22 last 13.68)
 - mkdir 696 (time 1788553707.17 total 50.22 last 14.80)
 - mkdir 875 (time 1788553717.22 total 60.28 last 17.80)
 - mkdir 992 (time 1788553727.23 total 70.29 last 11.69)
total: 1000 mkdir in 71.43 seconds: 14.00 ops/second
 - mkdir 126 (time 1788553666.73 total 10.11 last 12.47)
 - mkdir 278 (time 1788553676.77 total 20.15 last 15.13)
 - mkdir 413 (time 1788553686.79 total 30.17 last 13.47)
 - mkdir 549 (time 1788553696.89 total 40.27 last 13.47)
 - mkdir 692 (time 1788553706.92 total 50.30 last 14.26)
 - mkdir 871 (time 1788553716.94 total 60.32 last 17.86)
 - mkdir 988 (time 1788553727.14 total 70.51 last 11.47)
total: 1000 mkdir in 72.31 seconds: 13.83 ops/second
 - mkdir 123 (time 1788553666.09 total 10.00 last 12.30)
 - mkdir 273 (time 1788553676.17 total 20.08 last 14.88)
 - mkdir 401 (time 1788553686.20 total 30.11 last 12.76)
 - mkdir 538 (time 1788553696.21 total 40.12 last 13.69)
 - mkdir 682 (time 1788553706.28 total 50.19 last 14.30)
 - mkdir 856 (time 1788553716.33 total 60.24 last 17.31)
 - mkdir 977 (time 1788553726.40 total 70.31 last 12.03)
total: 1000 mkdir in 73.44 seconds: 13.62 ops/second
 - mkdir 121 (time 1788553667.22 total 10.03 last 12.07)
 - mkdir 281 (time 1788553677.26 total 20.06 last 15.95)
 - mkdir 419 (time 1788553687.31 total 30.12 last 13.72)
 - mkdir 551 (time 1788553697.39 total 40.20 last 13.10)
 - mkdir 698 (time 1788553707.41 total 50.22 last 14.67)
 - mkdir 877 (time 1788553717.45 total 60.25 last 17.84)
 - mkdir 987 (time 1788553727.60 total 70.40 last 10.84)
total: 1000 mkdir in 72.43 seconds: 13.81 ops/second
 - mkdir 121 (time 1788553666.48 total 10.01 last 12.09)
 - mkdir 274 (time 1788553676.60 total 20.13 last 15.12)
 - mkdir 406 (time 1788553686.66 total 30.19 last 13.12)
 - mkdir 542 (time 1788553696.70 total 40.23 last 13.55)
 - mkdir 687 (time 1788553706.73 total 50.26 last 14.45)
 - mkdir 863 (time 1788553716.76 total 60.28 last 17.56)
 - mkdir 980 (time 1788553726.83 total 70.36 last 11.62)
total: 1000 mkdir in 73.49 seconds: 13.61 ops/second
 - mkdir 116 (time 1788553665.35 total 10.04 last 11.55)
 - mkdir 266 (time 1788553675.39 total 20.08 last 14.94)
 - mkdir 387 (time 1788553685.41 total 30.11 last 12.07)
 - mkdir 523 (time 1788553695.43 total 40.12 last 13.58)
 - mkdir 667 (time 1788553705.43 total 50.13 last 14.39)
 - mkdir 837 (time 1788553715.48 total 60.17 last 16.93)
 - mkdir 960 (time 1788553725.49 total 70.18 last 12.28)
total: 1000 mkdir in 75.47 seconds: 13.25 ops/second
 - mkdir 126 (time 1788553667.36 total 10.04 last 12.55)
 - mkdir 279 (time 1788553677.37 total 20.04 last 15.29)
 - mkdir 411 (time 1788553687.43 total 30.11 last 13.12)
 - mkdir 540 (time 1788553697.46 total 40.13 last 12.86)
 - mkdir 681 (time 1788553707.46 total 50.14 last 14.10)
 - mkdir 857 (time 1788553717.54 total 60.21 last 17.47)
 - mkdir 966 (time 1788553727.69 total 70.36 last 10.74)
total: 1000 mkdir in 74.80 seconds: 13.37 ops/second
 - mkdir 112 (time 1788553666.14 total 10.03 last 11.17)
 - mkdir 259 (time 1788553676.23 total 20.11 last 14.58)
 - mkdir 382 (time 1788553686.26 total 30.15 last 12.26)
 - mkdir 514 (time 1788553696.29 total 40.18 last 13.16)
 - mkdir 647 (time 1788553706.33 total 50.21 last 13.25)
 - mkdir 817 (time 1788553716.39 total 60.28 last 16.89)
 - mkdir 934 (time 1788553726.39 total 70.28 last 11.70)
total: 1000 mkdir in 78.04 seconds: 12.81 ops/second
Time for file creation 91 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788553745 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1788553746 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1788553746 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1788553747 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1788553747 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1788553747 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1788553748 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1788553748 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1788553748 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1788553748 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
Time for file removal 55 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8