== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 22:53:12 (1787885592)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.60 seconds: 16.59 ops/second
 - mkdir 109 (time 1787885615.42 total 10.01 last 10.89)
 - mkdir 256 (time 1787885625.43 total 20.01 last 14.69)
 - mkdir 404 (time 1787885635.44 total 30.02 last 14.78)
 - mkdir 580 (time 1787885645.56 total 40.15 last 17.38)
 - mkdir 732 (time 1787885655.63 total 50.22 last 15.10)
 - mkdir 904 (time 1787885665.66 total 60.24 last 17.16)
total: 1000 mkdir in 65.26 seconds: 15.32 ops/second
 - mkdir 111 (time 1787885617.06 total 10.07 last 11.02)
 - mkdir 259 (time 1787885627.08 total 20.09 last 14.78)
 - mkdir 423 (time 1787885637.08 total 30.09 last 16.39)
 - mkdir 597 (time 1787885647.13 total 40.14 last 17.31)
 - mkdir 750 (time 1787885657.25 total 50.27 last 15.12)
 - mkdir 917 (time 1787885667.27 total 60.29 last 16.67)
total: 1000 mkdir in 64.64 seconds: 15.47 ops/second
 - mkdir 108 (time 1787885616.09 total 10.01 last 10.79)
 - mkdir 259 (time 1787885626.13 total 20.05 last 15.04)
 - mkdir 408 (time 1787885636.16 total 30.08 last 14.86)
 - mkdir 584 (time 1787885646.18 total 40.10 last 17.56)
 - mkdir 731 (time 1787885656.21 total 50.13 last 14.66)
 - mkdir 898 (time 1787885666.29 total 60.21 last 16.56)
total: 1000 mkdir in 66.47 seconds: 15.04 ops/second
 - mkdir 104 (time 1787885617.03 total 10.01 last 10.39)
 - mkdir 255 (time 1787885627.06 total 20.04 last 15.06)
 - mkdir 415 (time 1787885637.08 total 30.06 last 15.96)
 - mkdir 586 (time 1787885647.13 total 40.11 last 17.02)
 - mkdir 733 (time 1787885657.20 total 50.18 last 14.60)
 - mkdir 908 (time 1787885667.24 total 60.22 last 17.43)
total: 1000 mkdir in 65.84 seconds: 15.19 ops/second
 - mkdir 101 (time 1787885615.44 total 10.16 last 9.94)
 - mkdir 248 (time 1787885625.51 total 20.24 last 14.59)
 - mkdir 392 (time 1787885635.52 total 30.24 last 14.39)
 - mkdir 564 (time 1787885645.53 total 40.25 last 17.18)
 - mkdir 709 (time 1787885655.56 total 50.28 last 14.45)
 - mkdir 881 (time 1787885665.61 total 60.33 last 17.12)
total: 1000 mkdir in 68.38 seconds: 14.62 ops/second
 - mkdir 100 (time 1787885616.55 total 10.02 last 9.98)
 - mkdir 252 (time 1787885626.65 total 20.11 last 15.05)
 - mkdir 405 (time 1787885636.65 total 30.12 last 15.30)
 - mkdir 584 (time 1787885646.74 total 40.21 last 17.74)
 - mkdir 728 (time 1787885656.81 total 50.28 last 14.30)
 - mkdir 894 (time 1787885666.89 total 60.35 last 16.47)
total: 1000 mkdir in 67.31 seconds: 14.86 ops/second
 - mkdir 100 (time 1787885616.67 total 10.03 last 9.97)
 - mkdir 251 (time 1787885626.76 total 20.13 last 14.95)
 - mkdir 407 (time 1787885636.77 total 30.14 last 15.59)
 - mkdir 581 (time 1787885646.84 total 40.20 last 17.29)
 - mkdir 729 (time 1787885656.89 total 50.25 last 14.73)
 - mkdir 898 (time 1787885666.96 total 60.32 last 16.79)
total: 1000 mkdir in 67.51 seconds: 14.81 ops/second
 - mkdir 102 (time 1787885616.21 total 10.04 last 10.16)
 - mkdir 250 (time 1787885626.25 total 20.09 last 14.74)
 - mkdir 402 (time 1787885636.26 total 30.09 last 15.19)
 - mkdir 578 (time 1787885646.30 total 40.14 last 17.53)
 - mkdir 723 (time 1787885656.33 total 50.16 last 14.46)
 - mkdir 889 (time 1787885666.38 total 60.21 last 16.52)
total: 1000 mkdir in 68.22 seconds: 14.66 ops/second
 - mkdir 93 (time 1787885616.46 total 10.05 last 9.25)
 - mkdir 238 (time 1787885626.51 total 20.11 last 14.42)
 - mkdir 386 (time 1787885636.54 total 30.14 last 14.76)
 - mkdir 560 (time 1787885646.58 total 40.17 last 17.34)
 - mkdir 701 (time 1787885656.60 total 50.20 last 14.07)
 - mkdir 872 (time 1787885666.70 total 60.29 last 16.94)
total: 1000 mkdir in 70.18 seconds: 14.25 ops/second
 - mkdir 94 (time 1787885617.27 total 10.02 last 9.38)
 - mkdir 232 (time 1787885627.31 total 20.06 last 13.74)
 - mkdir 385 (time 1787885637.36 total 30.10 last 15.23)
 - mkdir 549 (time 1787885647.38 total 40.13 last 16.36)
 - mkdir 689 (time 1787885657.44 total 50.19 last 13.91)
 - mkdir 846 (time 1787885667.53 total 60.28 last 15.56)
 - mkdir 970 (time 1787885677.59 total 70.33 last 12.33)
total: 1000 mkdir in 71.86 seconds: 13.92 ops/second
Time for file creation 83 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 16 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.77 seconds: 12.93 ops/second
 - mkdir 105 (time 1787886274.76 total 10.02 last 10.48)
 - mkdir 233 (time 1787886284.77 total 20.02 last 12.80)
 - mkdir 370 (time 1787886294.79 total 30.04 last 13.67)
 - mkdir 510 (time 1787886304.80 total 40.05 last 13.99)
 - mkdir 633 (time 1787886314.83 total 50.08 last 12.26)
 - mkdir 799 (time 1787886324.84 total 60.09 last 16.59)
 - mkdir 950 (time 1787886334.86 total 70.12 last 15.06)
total: 1000 mkdir in 73.78 seconds: 13.55 ops/second
 - mkdir 113 (time 1787886276.17 total 10.04 last 11.26)
 - mkdir 244 (time 1787886286.20 total 20.07 last 13.06)
 - mkdir 385 (time 1787886296.20 total 30.07 last 14.10)
 - mkdir 524 (time 1787886306.22 total 40.09 last 13.87)
 - mkdir 653 (time 1787886316.22 total 50.09 last 12.90)
 - mkdir 814 (time 1787886326.29 total 60.16 last 16.00)
 - mkdir 964 (time 1787886336.37 total 70.24 last 14.88)
total: 1000 mkdir in 73.18 seconds: 13.66 ops/second
 - mkdir 109 (time 1787886276.32 total 10.02 last 10.88)
 - mkdir 237 (time 1787886286.37 total 20.06 last 12.75)
 - mkdir 379 (time 1787886296.45 total 30.14 last 14.09)
 - mkdir 515 (time 1787886306.48 total 40.17 last 13.56)
 - mkdir 640 (time 1787886316.50 total 50.19 last 12.48)
 - mkdir 802 (time 1787886326.51 total 60.20 last 16.18)
 - mkdir 952 (time 1787886336.60 total 70.29 last 14.86)
total: 1000 mkdir in 74.73 seconds: 13.38 ops/second
 - mkdir 101 (time 1787886275.62 total 10.07 last 10.03)
 - mkdir 225 (time 1787886285.64 total 20.09 last 12.37)
 - mkdir 363 (time 1787886295.64 total 30.10 last 13.79)
 - mkdir 500 (time 1787886305.67 total 40.13 last 13.66)
 - mkdir 622 (time 1787886315.70 total 50.15 last 12.17)
 - mkdir 791 (time 1787886325.70 total 60.15 last 16.90)
 - mkdir 944 (time 1787886335.78 total 70.23 last 15.18)
total: 1000 mkdir in 76.19 seconds: 13.12 ops/second
 - mkdir 118 (time 1787886277.25 total 10.07 last 11.72)
 - mkdir 243 (time 1787886287.34 total 20.17 last 12.38)
 - mkdir 384 (time 1787886297.34 total 30.17 last 14.10)
 - mkdir 521 (time 1787886307.39 total 40.22 last 13.64)
 - mkdir 652 (time 1787886317.41 total 50.24 last 13.07)
 - mkdir 812 (time 1787886327.45 total 60.27 last 15.94)
 - mkdir 960 (time 1787886337.50 total 70.33 last 14.72)
total: 1000 mkdir in 74.61 seconds: 13.40 ops/second
 - mkdir 112 (time 1787886277.17 total 10.10 last 11.09)
 - mkdir 234 (time 1787886287.22 total 20.16 last 12.13)
 - mkdir 374 (time 1787886297.26 total 30.19 last 13.95)
 - mkdir 509 (time 1787886307.29 total 40.23 last 13.45)
 - mkdir 639 (time 1787886317.31 total 50.25 last 12.98)
 - mkdir 800 (time 1787886327.31 total 60.25 last 16.10)
 - mkdir 950 (time 1787886337.35 total 70.29 last 14.94)
total: 1000 mkdir in 75.28 seconds: 13.28 ops/second
 - mkdir 109 (time 1787886276.98 total 10.10 last 10.79)
 - mkdir 228 (time 1787886287.01 total 20.13 last 11.86)
 - mkdir 367 (time 1787886297.01 total 30.14 last 13.90)
 - mkdir 507 (time 1787886307.07 total 40.19 last 13.92)
 - mkdir 636 (time 1787886317.08 total 50.21 last 12.88)
 - mkdir 795 (time 1787886327.08 total 60.21 last 15.90)
 - mkdir 942 (time 1787886337.09 total 70.21 last 14.70)
total: 1000 mkdir in 76.89 seconds: 13.01 ops/second
 - mkdir 110 (time 1787886276.78 total 10.07 last 10.92)
 - mkdir 234 (time 1787886286.80 total 20.10 last 12.37)
 - mkdir 370 (time 1787886296.92 total 30.22 last 13.44)
 - mkdir 511 (time 1787886307.03 total 40.32 last 13.95)
 - mkdir 637 (time 1787886317.03 total 50.33 last 12.59)
 - mkdir 795 (time 1787886327.05 total 60.35 last 15.77)
 - mkdir 940 (time 1787886337.08 total 70.38 last 14.46)
total: 1000 mkdir in 77.21 seconds: 12.95 ops/second
 - mkdir 112 (time 1787886277.36 total 10.10 last 11.09)
 - mkdir 234 (time 1787886287.38 total 20.12 last 12.18)
 - mkdir 365 (time 1787886297.41 total 30.15 last 13.06)
 - mkdir 507 (time 1787886307.43 total 40.17 last 14.17)
 - mkdir 636 (time 1787886317.45 total 50.19 last 12.87)
 - mkdir 794 (time 1787886327.53 total 60.26 last 15.68)
 - mkdir 932 (time 1787886337.58 total 70.32 last 13.73)
total: 1000 mkdir in 79.15 seconds: 12.63 ops/second
 - mkdir 101 (time 1787886276.96 total 10.02 last 10.08)
 - mkdir 222 (time 1787886286.98 total 20.04 last 12.08)
 - mkdir 353 (time 1787886297.01 total 30.08 last 13.05)
 - mkdir 489 (time 1787886307.07 total 40.13 last 13.52)
 - mkdir 613 (time 1787886317.08 total 50.14 last 12.39)
 - mkdir 770 (time 1787886327.11 total 60.17 last 15.65)
 - mkdir 904 (time 1787886337.12 total 70.18 last 13.39)
 - mkdir 988 (time 1787886347.20 total 80.26 last 8.33)
total: 1000 mkdir in 81.06 seconds: 12.34 ops/second
Time for file creation 95 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787886359 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787886359 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1787886360 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787886361 ; total 1 ; last 1)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787886360 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1787886361 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787886360 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1787886361 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1787886361 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1787886361 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
Time for file removal 54 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8