== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 07:38:00 (1790941080)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.51 seconds: 19.69 ops/second
 - mkdir 141 (time 1790941101.97 total 10.01 last 14.09)
 - mkdir 301 (time 1790941112.01 total 20.05 last 15.94)
 - mkdir 485 (time 1790941122.03 total 30.07 last 18.36)
 - mkdir 649 (time 1790941132.09 total 40.13 last 16.31)
 - mkdir 773 (time 1790941142.09 total 50.14 last 12.39)
 - mkdir 928 (time 1790941152.11 total 60.15 last 15.48)
total: 1000 mkdir in 64.51 seconds: 15.50 ops/second
 - mkdir 134 (time 1790941103.88 total 10.04 last 13.34)
 - mkdir 301 (time 1790941113.90 total 20.06 last 16.68)
 - mkdir 484 (time 1790941123.92 total 30.08 last 18.26)
 - mkdir 628 (time 1790941133.94 total 40.10 last 14.38)
 - mkdir 753 (time 1790941143.95 total 50.11 last 12.49)
 - mkdir 913 (time 1790941153.98 total 60.14 last 15.95)
total: 1000 mkdir in 69.85 seconds: 14.32 ops/second
 - mkdir 139 (time 1790941102.79 total 10.04 last 13.85)
 - mkdir 300 (time 1790941112.82 total 20.06 last 16.06)
 - mkdir 480 (time 1790941122.84 total 30.08 last 17.97)
 - mkdir 629 (time 1790941132.87 total 40.11 last 14.86)
 - mkdir 744 (time 1790941142.91 total 50.15 last 11.45)
 - mkdir 895 (time 1790941152.92 total 60.16 last 15.08)
 - mkdir 993 (time 1790941163.04 total 70.28 last 9.68)
total: 1000 mkdir in 71.22 seconds: 14.04 ops/second
 - mkdir 132 (time 1790941102.44 total 10.01 last 13.19)
 - mkdir 295 (time 1790941112.50 total 20.06 last 16.21)
 - mkdir 471 (time 1790941122.55 total 30.11 last 17.51)
 - mkdir 626 (time 1790941132.69 total 40.25 last 15.28)
 - mkdir 735 (time 1790941142.70 total 50.27 last 10.88)
 - mkdir 891 (time 1790941152.76 total 60.33 last 15.50)
 - mkdir 992 (time 1790941162.90 total 70.47 last 9.96)
total: 1000 mkdir in 71.60 seconds: 13.97 ops/second
 - mkdir 141 (time 1790941103.58 total 10.02 last 14.07)
 - mkdir 307 (time 1790941113.63 total 20.08 last 16.51)
 - mkdir 487 (time 1790941123.68 total 30.13 last 17.91)
 - mkdir 630 (time 1790941133.69 total 40.14 last 14.28)
 - mkdir 747 (time 1790941143.74 total 50.19 last 11.64)
 - mkdir 903 (time 1790941153.76 total 60.20 last 15.58)
 - mkdir 993 (time 1790941163.91 total 70.36 last 8.86)
total: 1000 mkdir in 71.42 seconds: 14.00 ops/second
 - mkdir 138 (time 1790941104.16 total 10.01 last 13.79)
 - mkdir 302 (time 1790941114.18 total 20.03 last 16.37)
 - mkdir 488 (time 1790941124.23 total 30.08 last 18.51)
 - mkdir 619 (time 1790941134.23 total 40.08 last 13.10)
 - mkdir 741 (time 1790941144.24 total 50.09 last 12.18)
 - mkdir 902 (time 1790941154.26 total 60.11 last 16.06)
 - mkdir 986 (time 1790941164.43 total 70.28 last 8.26)
total: 1000 mkdir in 72.07 seconds: 13.88 ops/second
 - mkdir 138 (time 1790941102.94 total 10.07 last 13.70)
 - mkdir 298 (time 1790941112.97 total 20.10 last 15.96)
 - mkdir 468 (time 1790941122.98 total 30.11 last 16.99)
 - mkdir 610 (time 1790941133.02 total 40.15 last 14.13)
 - mkdir 721 (time 1790941143.05 total 50.18 last 11.07)
 - mkdir 876 (time 1790941153.08 total 60.21 last 15.46)
 - mkdir 972 (time 1790941163.23 total 70.37 last 9.45)
total: 1000 mkdir in 73.98 seconds: 13.52 ops/second
 - mkdir 136 (time 1790941102.92 total 10.07 last 13.51)
 - mkdir 295 (time 1790941112.92 total 20.07 last 15.89)
 - mkdir 470 (time 1790941122.94 total 30.09 last 17.47)
 - mkdir 614 (time 1790941132.99 total 40.14 last 14.33)
 - mkdir 722 (time 1790941143.05 total 50.20 last 10.73)
 - mkdir 871 (time 1790941153.08 total 60.23 last 14.85)
 - mkdir 967 (time 1790941163.12 total 70.27 last 9.56)
total: 1000 mkdir in 74.31 seconds: 13.46 ops/second
 - mkdir 139 (time 1790941103.96 total 10.09 last 13.78)
 - mkdir 299 (time 1790941113.97 total 20.09 last 15.99)
 - mkdir 479 (time 1790941123.98 total 30.11 last 17.97)
 - mkdir 612 (time 1790941133.99 total 40.11 last 13.30)
 - mkdir 734 (time 1790941144.01 total 50.14 last 12.16)
 - mkdir 889 (time 1790941154.04 total 60.17 last 15.46)
 - mkdir 974 (time 1790941164.06 total 70.19 last 8.48)
total: 1000 mkdir in 73.34 seconds: 13.64 ops/second
 - mkdir 126 (time 1790941101.74 total 10.02 last 12.58)
 - mkdir 271 (time 1790941111.75 total 20.03 last 14.48)
 - mkdir 434 (time 1790941121.75 total 30.03 last 16.30)
 - mkdir 581 (time 1790941131.82 total 40.10 last 14.60)
 - mkdir 693 (time 1790941141.84 total 50.12 last 11.18)
 - mkdir 834 (time 1790941151.87 total 60.15 last 14.06)
 - mkdir 944 (time 1790941162.08 total 70.36 last 10.78)
total: 1000 mkdir in 76.70 seconds: 13.04 ops/second
Time for file creation 86 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 14 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.40 seconds: 24.80 ops/second
 - mkdir 177 (time 1790941688.31 total 10.04 last 17.63)
 - mkdir 395 (time 1790941698.34 total 20.07 last 21.73)
 - mkdir 568 (time 1790941708.35 total 30.08 last 17.28)
 - mkdir 765 (time 1790941718.38 total 40.12 last 19.63)
 - mkdir 939 (time 1790941728.39 total 50.12 last 17.39)
total: 1000 mkdir in 54.02 seconds: 18.51 ops/second
 - mkdir 172 (time 1790941688.33 total 10.06 last 17.10)
 - mkdir 392 (time 1790941698.35 total 20.08 last 21.96)
 - mkdir 564 (time 1790941708.38 total 30.10 last 17.16)
 - mkdir 751 (time 1790941718.45 total 40.17 last 18.57)
 - mkdir 922 (time 1790941728.46 total 50.18 last 17.08)
total: 1000 mkdir in 55.41 seconds: 18.05 ops/second
 - mkdir 173 (time 1790941688.34 total 10.02 last 17.26)
 - mkdir 386 (time 1790941698.34 total 20.02 last 21.29)
 - mkdir 553 (time 1790941708.34 total 30.03 last 16.69)
 - mkdir 745 (time 1790941718.36 total 40.04 last 19.18)
 - mkdir 914 (time 1790941728.36 total 50.05 last 16.89)
total: 1000 mkdir in 56.31 seconds: 17.76 ops/second
 - mkdir 172 (time 1790941688.82 total 10.05 last 17.12)
 - mkdir 389 (time 1790941698.83 total 20.06 last 21.68)
 - mkdir 561 (time 1790941708.83 total 30.06 last 17.19)
 - mkdir 751 (time 1790941718.89 total 40.12 last 18.89)
 - mkdir 921 (time 1790941728.95 total 50.18 last 16.90)
total: 1000 mkdir in 56.00 seconds: 17.86 ops/second
 - mkdir 176 (time 1790941688.46 total 10.06 last 17.49)
 - mkdir 387 (time 1790941698.51 total 20.11 last 21.00)
 - mkdir 556 (time 1790941708.55 total 30.15 last 16.83)
 - mkdir 742 (time 1790941718.61 total 40.21 last 18.49)
 - mkdir 912 (time 1790941728.64 total 50.24 last 16.96)
total: 1000 mkdir in 56.57 seconds: 17.68 ops/second
 - mkdir 175 (time 1790941689.14 total 10.04 last 17.43)
 - mkdir 388 (time 1790941699.14 total 20.05 last 21.29)
 - mkdir 560 (time 1790941709.17 total 30.07 last 17.15)
 - mkdir 748 (time 1790941719.17 total 40.08 last 18.80)
 - mkdir 916 (time 1790941729.24 total 50.14 last 16.69)
total: 1000 mkdir in 56.68 seconds: 17.64 ops/second
 - mkdir 170 (time 1790941687.78 total 10.06 last 16.89)
 - mkdir 373 (time 1790941697.82 total 20.10 last 20.23)
 - mkdir 542 (time 1790941707.88 total 30.16 last 16.80)
 - mkdir 726 (time 1790941717.90 total 40.18 last 18.36)
 - mkdir 893 (time 1790941727.95 total 50.23 last 16.62)
total: 1000 mkdir in 58.43 seconds: 17.11 ops/second
 - mkdir 168 (time 1790941688.56 total 10.01 last 16.79)
 - mkdir 380 (time 1790941698.57 total 20.02 last 21.17)
 - mkdir 546 (time 1790941708.60 total 30.05 last 16.55)
 - mkdir 733 (time 1790941718.62 total 40.07 last 18.66)
 - mkdir 900 (time 1790941728.65 total 50.10 last 16.64)
total: 1000 mkdir in 58.24 seconds: 17.17 ops/second
 - mkdir 165 (time 1790941689.30 total 10.05 last 16.42)
 - mkdir 382 (time 1790941699.32 total 20.07 last 21.66)
 - mkdir 556 (time 1790941709.32 total 30.07 last 17.40)
 - mkdir 734 (time 1790941719.36 total 40.10 last 17.74)
 - mkdir 904 (time 1790941729.46 total 50.21 last 16.82)
total: 1000 mkdir in 58.16 seconds: 17.19 ops/second
 - mkdir 166 (time 1790941689.04 total 10.06 last 16.51)
 - mkdir 373 (time 1790941699.05 total 20.06 last 20.68)
 - mkdir 533 (time 1790941709.10 total 30.12 last 15.92)
 - mkdir 713 (time 1790941719.15 total 40.17 last 17.90)
 - mkdir 875 (time 1790941729.22 total 50.23 last 16.10)
 - mkdir 994 (time 1790941739.25 total 60.26 last 11.86)
total: 1000 mkdir in 60.44 seconds: 16.55 ops/second
Time for file creation 69 sec for 1 RPC sent at a time
 - unlinked 0 (time 1790941747 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1790941747 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1790941748 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1790941749 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1790941749 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1790941750 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1790941750 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1790941750 ; total 0 ; last 0)
total: 1000 unlinks in 29 seconds: 34.482758 unlinks/second
 - unlinked 0 (time 1790941749 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1790941749 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
Time for file removal 42 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8