== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 04:44:24 (1788857064)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.54 seconds: 18.64 ops/second
 - mkdir 142 (time 1788857085.69 total 10.01 last 14.19)
 - mkdir 289 (time 1788857095.76 total 20.08 last 14.59)
 - mkdir 413 (time 1788857105.80 total 30.12 last 12.35)
 - mkdir 569 (time 1788857115.87 total 40.19 last 15.49)
 - mkdir 711 (time 1788857125.90 total 50.22 last 14.16)
 - mkdir 838 (time 1788857135.90 total 60.22 last 12.69)
 - mkdir 971 (time 1788857145.95 total 70.27 last 13.24)
total: 1000 mkdir in 72.00 seconds: 13.89 ops/second
 - mkdir 138 (time 1788857086.21 total 10.01 last 13.78)
 - mkdir 288 (time 1788857096.29 total 20.10 last 14.88)
 - mkdir 415 (time 1788857106.35 total 30.16 last 12.63)
 - mkdir 567 (time 1788857116.39 total 40.20 last 15.14)
 - mkdir 706 (time 1788857126.43 total 50.24 last 13.84)
 - mkdir 830 (time 1788857136.45 total 60.26 last 12.37)
 - mkdir 963 (time 1788857146.46 total 70.27 last 13.29)
total: 1000 mkdir in 73.63 seconds: 13.58 ops/second
 - mkdir 142 (time 1788857087.14 total 10.01 last 14.19)
 - mkdir 288 (time 1788857097.16 total 20.02 last 14.57)
 - mkdir 414 (time 1788857107.18 total 30.04 last 12.58)
 - mkdir 575 (time 1788857117.22 total 40.08 last 16.04)
 - mkdir 708 (time 1788857127.25 total 50.11 last 13.26)
 - mkdir 843 (time 1788857137.25 total 60.11 last 13.50)
 - mkdir 975 (time 1788857147.26 total 70.12 last 13.19)
total: 1000 mkdir in 73.57 seconds: 13.59 ops/second
 - mkdir 136 (time 1788857087.96 total 10.04 last 13.54)
 - mkdir 283 (time 1788857097.98 total 20.07 last 14.67)
 - mkdir 404 (time 1788857108.08 total 30.17 last 11.98)
 - mkdir 571 (time 1788857118.11 total 40.20 last 16.65)
 - mkdir 707 (time 1788857128.18 total 50.27 last 13.51)
 - mkdir 840 (time 1788857138.21 total 60.29 last 13.26)
 - mkdir 968 (time 1788857148.38 total 70.46 last 12.59)
total: 1000 mkdir in 74.92 seconds: 13.35 ops/second
 - mkdir 141 (time 1788857088.10 total 10.08 last 13.99)
 - mkdir 288 (time 1788857098.11 total 20.09 last 14.68)
 - mkdir 410 (time 1788857108.12 total 30.10 last 12.19)
 - mkdir 574 (time 1788857118.15 total 40.13 last 16.35)
 - mkdir 705 (time 1788857128.25 total 50.23 last 12.97)
 - mkdir 840 (time 1788857138.30 total 60.28 last 13.43)
 - mkdir 966 (time 1788857148.36 total 70.35 last 12.52)
total: 1000 mkdir in 75.28 seconds: 13.28 ops/second
 - mkdir 136 (time 1788857087.82 total 10.06 last 13.52)
 - mkdir 282 (time 1788857097.86 total 20.10 last 14.54)
 - mkdir 405 (time 1788857107.87 total 30.11 last 12.29)
 - mkdir 574 (time 1788857117.88 total 40.12 last 16.88)
 - mkdir 701 (time 1788857127.90 total 50.14 last 12.68)
 - mkdir 832 (time 1788857137.91 total 60.15 last 13.08)
 - mkdir 964 (time 1788857147.91 total 70.15 last 13.20)
total: 1000 mkdir in 75.77 seconds: 13.20 ops/second
 - mkdir 136 (time 1788857087.93 total 10.08 last 13.49)
 - mkdir 287 (time 1788857097.98 total 20.12 last 15.03)
 - mkdir 411 (time 1788857108.00 total 30.15 last 12.37)
 - mkdir 573 (time 1788857118.06 total 40.20 last 16.11)
 - mkdir 704 (time 1788857128.12 total 50.26 last 13.02)
 - mkdir 837 (time 1788857138.13 total 60.27 last 13.28)
 - mkdir 964 (time 1788857148.32 total 70.46 last 12.46)
total: 1000 mkdir in 76.13 seconds: 13.13 ops/second
 - mkdir 136 (time 1788857088.21 total 10.00 last 13.60)
 - mkdir 283 (time 1788857098.21 total 20.01 last 14.69)
 - mkdir 404 (time 1788857108.21 total 30.01 last 12.10)
 - mkdir 572 (time 1788857118.27 total 40.07 last 16.70)
 - mkdir 702 (time 1788857128.32 total 50.11 last 12.94)
 - mkdir 836 (time 1788857138.39 total 60.18 last 13.30)
 - mkdir 965 (time 1788857148.43 total 70.22 last 12.85)
total: 1000 mkdir in 76.06 seconds: 13.15 ops/second
 - mkdir 131 (time 1788857087.52 total 10.03 last 13.07)
 - mkdir 272 (time 1788857097.59 total 20.10 last 13.99)
 - mkdir 390 (time 1788857107.63 total 30.14 last 11.76)
 - mkdir 556 (time 1788857117.64 total 40.15 last 16.59)
 - mkdir 686 (time 1788857127.64 total 50.15 last 12.99)
 - mkdir 819 (time 1788857137.66 total 60.17 last 13.29)
 - mkdir 950 (time 1788857147.68 total 70.19 last 13.07)
total: 1000 mkdir in 78.08 seconds: 12.81 ops/second
 - mkdir 132 (time 1788857088.73 total 10.10 last 13.08)
 - mkdir 275 (time 1788857098.78 total 20.15 last 14.22)
 - mkdir 397 (time 1788857108.79 total 30.16 last 12.19)
 - mkdir 568 (time 1788857118.84 total 40.21 last 17.02)
 - mkdir 694 (time 1788857128.94 total 50.31 last 12.47)
 - mkdir 832 (time 1788857138.96 total 60.33 last 13.77)
 - mkdir 952 (time 1788857149.13 total 70.50 last 11.80)
total: 1000 mkdir in 77.36 seconds: 12.93 ops/second
Time for file creation 89 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 14 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.40 seconds: 25.25 ops/second
 - mkdir 205 (time 1788857706.92 total 10.05 last 20.40)
 - mkdir 391 (time 1788857716.93 total 20.06 last 18.59)
 - mkdir 555 (time 1788857727.02 total 30.15 last 16.25)
 - mkdir 723 (time 1788857737.04 total 40.17 last 16.76)
 - mkdir 892 (time 1788857747.12 total 50.25 last 16.77)
total: 1000 mkdir in 56.63 seconds: 17.66 ops/second
 - mkdir 199 (time 1788857707.02 total 10.05 last 19.80)
 - mkdir 383 (time 1788857717.02 total 20.05 last 18.40)
 - mkdir 546 (time 1788857727.07 total 30.10 last 16.23)
 - mkdir 710 (time 1788857737.11 total 40.14 last 16.33)
 - mkdir 869 (time 1788857747.13 total 50.16 last 15.87)
total: 1000 mkdir in 59.39 seconds: 16.84 ops/second
 - mkdir 200 (time 1788857707.06 total 10.04 last 19.92)
 - mkdir 384 (time 1788857717.11 total 20.09 last 18.30)
 - mkdir 542 (time 1788857727.15 total 30.14 last 15.73)
 - mkdir 704 (time 1788857737.19 total 40.17 last 16.15)
 - mkdir 871 (time 1788857747.24 total 50.22 last 16.61)
total: 1000 mkdir in 59.55 seconds: 16.79 ops/second
 - mkdir 198 (time 1788857707.65 total 10.02 last 19.77)
 - mkdir 381 (time 1788857717.65 total 20.02 last 18.29)
 - mkdir 541 (time 1788857727.70 total 30.07 last 15.93)
 - mkdir 712 (time 1788857737.74 total 40.10 last 17.04)
 - mkdir 875 (time 1788857747.81 total 50.18 last 16.18)
total: 1000 mkdir in 59.00 seconds: 16.95 ops/second
 - mkdir 195 (time 1788857707.41 total 10.03 last 19.45)
 - mkdir 377 (time 1788857717.43 total 20.05 last 18.16)
 - mkdir 535 (time 1788857727.45 total 30.07 last 15.77)
 - mkdir 701 (time 1788857737.46 total 40.08 last 16.58)
 - mkdir 863 (time 1788857747.47 total 50.09 last 16.18)
total: 1000 mkdir in 59.62 seconds: 16.77 ops/second
 - mkdir 199 (time 1788857707.80 total 10.04 last 19.82)
 - mkdir 383 (time 1788857717.87 total 20.11 last 18.27)
 - mkdir 542 (time 1788857727.90 total 30.15 last 15.84)
 - mkdir 709 (time 1788857737.97 total 40.21 last 16.59)
 - mkdir 872 (time 1788857748.02 total 50.26 last 16.22)
total: 1000 mkdir in 59.62 seconds: 16.77 ops/second
 - mkdir 198 (time 1788857706.99 total 10.01 last 19.78)
 - mkdir 380 (time 1788857717.09 total 20.11 last 18.02)
 - mkdir 540 (time 1788857727.13 total 30.15 last 15.94)
 - mkdir 690 (time 1788857737.16 total 40.18 last 14.96)
 - mkdir 851 (time 1788857747.17 total 50.19 last 16.08)
 - mkdir 987 (time 1788857757.35 total 60.37 last 13.36)
total: 1000 mkdir in 61.91 seconds: 16.15 ops/second
 - mkdir 199 (time 1788857707.35 total 10.02 last 19.87)
 - mkdir 372 (time 1788857717.37 total 20.03 last 17.27)
 - mkdir 527 (time 1788857727.46 total 30.12 last 15.36)
 - mkdir 693 (time 1788857737.46 total 40.13 last 16.59)
 - mkdir 850 (time 1788857747.48 total 50.14 last 15.68)
 - mkdir 984 (time 1788857757.59 total 60.26 last 13.24)
total: 1000 mkdir in 61.98 seconds: 16.13 ops/second
 - mkdir 195 (time 1788857707.88 total 10.03 last 19.43)
 - mkdir 381 (time 1788857717.90 total 20.05 last 18.57)
 - mkdir 534 (time 1788857727.91 total 30.06 last 15.28)
 - mkdir 701 (time 1788857737.99 total 40.14 last 16.57)
 - mkdir 859 (time 1788857748.01 total 50.16 last 15.77)
 - mkdir 986 (time 1788857758.02 total 60.17 last 12.69)
total: 1000 mkdir in 61.82 seconds: 16.18 ops/second
 - mkdir 194 (time 1788857706.87 total 10.04 last 19.33)
 - mkdir 370 (time 1788857716.91 total 20.08 last 17.53)
 - mkdir 527 (time 1788857726.94 total 30.11 last 15.65)
 - mkdir 680 (time 1788857736.99 total 40.16 last 15.21)
 - mkdir 835 (time 1788857747.08 total 50.25 last 15.36)
 - mkdir 973 (time 1788857757.22 total 60.39 last 13.61)
total: 1000 mkdir in 63.32 seconds: 15.79 ops/second
Time for file creation 69 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788857766 ; total 0 ; last 0)
total: 1000 unlinks in 27 seconds: 37.037037 unlinks/second
 - unlinked 0 (time 1788857767 ; total 0 ; last 0)
total: 1000 unlinks in 27 seconds: 37.037037 unlinks/second
 - unlinked 0 (time 1788857767 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788857768 ; total 0 ; last 0)
total: 1000 unlinks in 27 seconds: 37.037037 unlinks/second
 - unlinked 0 (time 1788857768 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788857768 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788857768 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788857768 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1788857769 ; total 1 ; last 1)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1788857769 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
Time for file removal 38 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8