== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 23:14:47 (1788664487)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.54 seconds: 18.49 ops/second
 - mkdir 154 (time 1788664508.70 total 10.06 last 15.31)
 - mkdir 321 (time 1788664518.71 total 20.07 last 16.67)
 - mkdir 494 (time 1788664528.71 total 30.08 last 17.30)
 - mkdir 661 (time 1788664538.76 total 40.12 last 16.62)
 - mkdir 829 (time 1788664548.79 total 50.15 last 16.76)
 - mkdir 984 (time 1788664558.83 total 60.19 last 15.43)
total: 1000 mkdir in 61.16 seconds: 16.35 ops/second
 - mkdir 170 (time 1788664511.08 total 10.05 last 16.91)
 - mkdir 346 (time 1788664521.12 total 20.09 last 17.54)
 - mkdir 515 (time 1788664531.16 total 30.13 last 16.83)
 - mkdir 683 (time 1788664541.19 total 40.16 last 16.75)
 - mkdir 846 (time 1788664551.24 total 50.21 last 16.22)
 - mkdir 990 (time 1788664561.25 total 60.22 last 14.38)
total: 1000 mkdir in 61.45 seconds: 16.27 ops/second
 - mkdir 172 (time 1788664511.24 total 10.01 last 17.18)
 - mkdir 340 (time 1788664521.34 total 20.12 last 16.63)
 - mkdir 508 (time 1788664531.40 total 30.18 last 16.69)
 - mkdir 679 (time 1788664541.43 total 40.20 last 17.06)
 - mkdir 841 (time 1788664551.49 total 50.27 last 16.10)
 - mkdir 988 (time 1788664561.60 total 60.37 last 14.54)
total: 1000 mkdir in 61.77 seconds: 16.19 ops/second
 - mkdir 172 (time 1788664512.08 total 10.04 last 17.14)
 - mkdir 339 (time 1788664522.11 total 20.06 last 16.66)
 - mkdir 506 (time 1788664532.14 total 30.09 last 16.65)
 - mkdir 681 (time 1788664542.16 total 40.11 last 17.47)
 - mkdir 843 (time 1788664552.18 total 50.13 last 16.15)
 - mkdir 986 (time 1788664562.19 total 60.14 last 14.29)
total: 1000 mkdir in 62.41 seconds: 16.02 ops/second
 - mkdir 168 (time 1788664512.21 total 10.01 last 16.78)
 - mkdir 341 (time 1788664522.23 total 20.03 last 17.27)
 - mkdir 512 (time 1788664532.24 total 30.04 last 17.07)
 - mkdir 678 (time 1788664542.30 total 40.10 last 16.50)
 - mkdir 839 (time 1788664552.33 total 50.13 last 16.06)
 - mkdir 978 (time 1788664562.36 total 60.16 last 13.85)
total: 1000 mkdir in 63.39 seconds: 15.78 ops/second
 - mkdir 168 (time 1788664511.73 total 10.02 last 16.77)
 - mkdir 338 (time 1788664521.74 total 20.03 last 16.98)
 - mkdir 507 (time 1788664531.81 total 30.09 last 16.79)
 - mkdir 675 (time 1788664541.85 total 40.14 last 16.72)
 - mkdir 832 (time 1788664551.91 total 50.19 last 15.61)
 - mkdir 976 (time 1788664561.94 total 60.22 last 14.36)
total: 1000 mkdir in 63.99 seconds: 15.63 ops/second
 - mkdir 171 (time 1788664511.71 total 10.05 last 17.01)
 - mkdir 336 (time 1788664521.74 total 20.07 last 16.46)
 - mkdir 500 (time 1788664531.74 total 30.08 last 16.39)
 - mkdir 668 (time 1788664541.77 total 40.11 last 16.74)
 - mkdir 825 (time 1788664551.78 total 50.12 last 15.69)
 - mkdir 969 (time 1788664561.82 total 60.15 last 14.34)
total: 1000 mkdir in 64.30 seconds: 15.55 ops/second
 - mkdir 163 (time 1788664512.07 total 10.04 last 16.23)
 - mkdir 332 (time 1788664522.10 total 20.08 last 16.85)
 - mkdir 502 (time 1788664532.16 total 30.14 last 16.90)
 - mkdir 669 (time 1788664542.23 total 40.21 last 16.59)
 - mkdir 827 (time 1788664552.26 total 50.24 last 15.75)
 - mkdir 969 (time 1788664562.35 total 60.33 last 14.07)
total: 1000 mkdir in 64.69 seconds: 15.46 ops/second
 - mkdir 163 (time 1788664512.09 total 10.00 last 16.29)
 - mkdir 320 (time 1788664522.12 total 20.03 last 15.66)
 - mkdir 484 (time 1788664532.15 total 30.06 last 16.35)
 - mkdir 651 (time 1788664542.22 total 40.13 last 16.58)
 - mkdir 814 (time 1788664552.25 total 50.16 last 16.24)
 - mkdir 960 (time 1788664562.30 total 60.21 last 14.53)
total: 1000 mkdir in 65.24 seconds: 15.33 ops/second
 - mkdir 156 (time 1788664511.47 total 10.01 last 15.58)
 - mkdir 320 (time 1788664521.51 total 20.06 last 16.33)
 - mkdir 483 (time 1788664531.56 total 30.11 last 16.22)
 - mkdir 650 (time 1788664541.56 total 40.11 last 16.70)
 - mkdir 808 (time 1788664551.58 total 50.13 last 15.77)
 - mkdir 956 (time 1788664561.65 total 60.19 last 14.70)
total: 1000 mkdir in 65.93 seconds: 15.17 ops/second
Time for file creation 78 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 14 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.71 seconds: 14.05 ops/second
 - mkdir 147 (time 1788665115.82 total 10.00 last 14.70)
 - mkdir 313 (time 1788665125.85 total 20.04 last 16.54)
 - mkdir 455 (time 1788665135.88 total 30.06 last 14.16)
 - mkdir 614 (time 1788665145.94 total 40.12 last 15.81)
 - mkdir 750 (time 1788665155.95 total 50.14 last 13.58)
 - mkdir 892 (time 1788665166.02 total 60.20 last 14.11)
total: 1000 mkdir in 67.79 seconds: 14.75 ops/second
 - mkdir 155 (time 1788665117.37 total 10.01 last 15.49)
 - mkdir 313 (time 1788665127.42 total 20.06 last 15.71)
 - mkdir 463 (time 1788665137.48 total 30.12 last 14.92)
 - mkdir 622 (time 1788665147.55 total 40.19 last 15.79)
 - mkdir 749 (time 1788665157.55 total 50.19 last 12.70)
 - mkdir 901 (time 1788665167.58 total 60.23 last 15.15)
total: 1000 mkdir in 67.24 seconds: 14.87 ops/second
 - mkdir 157 (time 1788665117.65 total 10.03 last 15.65)
 - mkdir 313 (time 1788665127.72 total 20.10 last 15.50)
 - mkdir 458 (time 1788665137.77 total 30.14 last 14.43)
 - mkdir 617 (time 1788665147.80 total 40.18 last 15.84)
 - mkdir 742 (time 1788665157.85 total 50.23 last 12.44)
 - mkdir 897 (time 1788665167.91 total 60.29 last 15.42)
total: 1000 mkdir in 69.02 seconds: 14.49 ops/second
 - mkdir 155 (time 1788665117.52 total 10.05 last 15.43)
 - mkdir 310 (time 1788665127.55 total 20.08 last 15.45)
 - mkdir 453 (time 1788665137.59 total 30.12 last 14.25)
 - mkdir 611 (time 1788665147.64 total 40.16 last 15.73)
 - mkdir 738 (time 1788665157.74 total 50.27 last 12.57)
 - mkdir 896 (time 1788665167.83 total 60.36 last 15.66)
total: 1000 mkdir in 69.33 seconds: 14.42 ops/second
 - mkdir 150 (time 1788665117.53 total 10.05 last 14.93)
 - mkdir 303 (time 1788665127.55 total 20.06 last 15.28)
 - mkdir 450 (time 1788665137.56 total 30.08 last 14.68)
 - mkdir 612 (time 1788665147.61 total 40.12 last 16.12)
 - mkdir 738 (time 1788665157.68 total 50.19 last 12.51)
 - mkdir 889 (time 1788665167.71 total 60.23 last 15.04)
total: 1000 mkdir in 70.00 seconds: 14.29 ops/second
 - mkdir 158 (time 1788665118.19 total 10.09 last 15.66)
 - mkdir 319 (time 1788665128.19 total 20.09 last 16.10)
 - mkdir 462 (time 1788665138.22 total 30.12 last 14.26)
 - mkdir 623 (time 1788665148.30 total 40.20 last 15.98)
 - mkdir 750 (time 1788665158.31 total 50.21 last 12.68)
 - mkdir 898 (time 1788665168.31 total 60.21 last 14.80)
total: 1000 mkdir in 70.16 seconds: 14.25 ops/second
 - mkdir 150 (time 1788665117.47 total 10.01 last 14.99)
 - mkdir 302 (time 1788665127.55 total 20.08 last 15.09)
 - mkdir 447 (time 1788665137.55 total 30.09 last 14.50)
 - mkdir 603 (time 1788665147.57 total 40.11 last 15.57)
 - mkdir 727 (time 1788665157.59 total 50.13 last 12.37)
 - mkdir 882 (time 1788665167.62 total 60.16 last 15.46)
 - mkdir 995 (time 1788665177.68 total 70.22 last 11.23)
total: 1000 mkdir in 70.80 seconds: 14.12 ops/second
 - mkdir 146 (time 1788665116.77 total 10.03 last 14.56)
 - mkdir 298 (time 1788665126.84 total 20.10 last 15.10)
 - mkdir 437 (time 1788665136.87 total 30.12 last 13.87)
 - mkdir 593 (time 1788665146.89 total 40.14 last 15.57)
 - mkdir 726 (time 1788665156.93 total 50.19 last 13.24)
 - mkdir 877 (time 1788665166.96 total 60.21 last 15.07)
 - mkdir 990 (time 1788665176.97 total 70.23 last 11.28)
total: 1000 mkdir in 71.62 seconds: 13.96 ops/second
 - mkdir 151 (time 1788665118.09 total 10.02 last 15.07)
 - mkdir 310 (time 1788665128.12 total 20.06 last 15.84)
 - mkdir 454 (time 1788665138.15 total 30.09 last 14.36)
 - mkdir 611 (time 1788665148.21 total 40.14 last 15.61)
 - mkdir 741 (time 1788665158.27 total 50.21 last 12.91)
 - mkdir 891 (time 1788665168.30 total 60.23 last 14.97)
 - mkdir 995 (time 1788665178.33 total 70.27 last 10.36)
total: 1000 mkdir in 70.75 seconds: 14.13 ops/second
 - mkdir 141 (time 1788665117.40 total 10.04 last 14.05)
 - mkdir 288 (time 1788665127.42 total 20.06 last 14.67)
 - mkdir 427 (time 1788665137.48 total 30.11 last 13.83)
 - mkdir 585 (time 1788665147.50 total 40.13 last 15.77)
 - mkdir 714 (time 1788665157.52 total 50.16 last 12.87)
 - mkdir 866 (time 1788665167.58 total 60.21 last 15.11)
 - mkdir 974 (time 1788665177.68 total 70.32 last 10.69)
total: 1000 mkdir in 73.19 seconds: 13.66 ops/second
Time for file creation 83 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788665190 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788665190 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788665190 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788665191 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788665191 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788665190 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788665191 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788665191 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788665191 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788665192 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
Time for file removal 45 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8