== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 07:33:55 (1787571235)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.81 seconds: 12.35 ops/second
 - mkdir 153 (time 1787571257.75 total 10.04 last 15.23)
 - mkdir 275 (time 1787571267.75 total 20.05 last 12.19)
 - mkdir 406 (time 1787571277.80 total 30.09 last 13.05)
 - mkdir 549 (time 1787571287.85 total 40.15 last 14.22)
 - mkdir 701 (time 1787571297.87 total 50.17 last 15.17)
 - mkdir 847 (time 1787571307.94 total 60.23 last 14.50)
 - mkdir 982 (time 1787571317.95 total 70.25 last 13.48)
total: 1000 mkdir in 71.62 seconds: 13.96 ops/second
 - mkdir 146 (time 1787571259.06 total 10.02 last 14.57)
 - mkdir 263 (time 1787571269.09 total 20.05 last 11.67)
 - mkdir 403 (time 1787571279.10 total 30.06 last 13.98)
 - mkdir 549 (time 1787571289.16 total 40.12 last 14.52)
 - mkdir 702 (time 1787571299.19 total 50.15 last 15.24)
 - mkdir 860 (time 1787571309.20 total 60.16 last 15.78)
 - mkdir 995 (time 1787571319.27 total 70.24 last 13.40)
total: 1000 mkdir in 70.82 seconds: 14.12 ops/second
 - mkdir 139 (time 1787571258.11 total 10.04 last 13.84)
 - mkdir 255 (time 1787571268.17 total 20.10 last 11.53)
 - mkdir 386 (time 1787571278.18 total 30.11 last 13.09)
 - mkdir 534 (time 1787571288.28 total 40.22 last 14.64)
 - mkdir 680 (time 1787571298.33 total 50.26 last 14.54)
 - mkdir 830 (time 1787571308.36 total 60.29 last 14.96)
 - mkdir 965 (time 1787571318.39 total 70.32 last 13.46)
total: 1000 mkdir in 74.44 seconds: 13.43 ops/second
 - mkdir 142 (time 1787571258.65 total 10.03 last 14.16)
 - mkdir 256 (time 1787571268.67 total 20.04 last 11.38)
 - mkdir 389 (time 1787571278.78 total 30.15 last 13.16)
 - mkdir 533 (time 1787571288.80 total 40.17 last 14.37)
 - mkdir 685 (time 1787571298.83 total 50.20 last 15.16)
 - mkdir 836 (time 1787571308.91 total 60.28 last 14.98)
 - mkdir 971 (time 1787571318.96 total 70.33 last 13.43)
total: 1000 mkdir in 74.22 seconds: 13.47 ops/second
 - mkdir 143 (time 1787571258.26 total 10.02 last 14.28)
 - mkdir 256 (time 1787571268.32 total 20.07 last 11.24)
 - mkdir 392 (time 1787571278.35 total 30.11 last 13.55)
 - mkdir 529 (time 1787571288.39 total 40.14 last 13.65)
 - mkdir 683 (time 1787571298.43 total 50.18 last 15.34)
 - mkdir 828 (time 1787571308.46 total 60.21 last 14.45)
 - mkdir 961 (time 1787571318.47 total 70.22 last 13.29)
total: 1000 mkdir in 75.54 seconds: 13.24 ops/second
 - mkdir 146 (time 1787571257.00 total 10.02 last 14.58)
 - mkdir 259 (time 1787571267.05 total 20.07 last 11.24)
 - mkdir 385 (time 1787571277.10 total 30.11 last 12.55)
 - mkdir 518 (time 1787571287.16 total 40.17 last 13.22)
 - mkdir 666 (time 1787571297.18 total 50.20 last 14.77)
 - mkdir 810 (time 1787571307.23 total 60.24 last 14.33)
 - mkdir 943 (time 1787571317.26 total 70.27 last 13.26)
total: 1000 mkdir in 77.17 seconds: 12.96 ops/second
 - mkdir 142 (time 1787571259.03 total 10.01 last 14.19)
 - mkdir 259 (time 1787571269.09 total 20.06 last 11.64)
 - mkdir 393 (time 1787571279.14 total 30.11 last 13.33)
 - mkdir 542 (time 1787571289.20 total 40.18 last 14.81)
 - mkdir 692 (time 1787571299.20 total 50.18 last 14.99)
 - mkdir 838 (time 1787571309.23 total 60.21 last 14.55)
 - mkdir 970 (time 1787571319.33 total 70.30 last 13.08)
total: 1000 mkdir in 75.16 seconds: 13.31 ops/second
 - mkdir 136 (time 1787571258.81 total 10.04 last 13.54)
 - mkdir 250 (time 1787571268.87 total 20.10 last 11.34)
 - mkdir 386 (time 1787571278.90 total 30.13 last 13.56)
 - mkdir 530 (time 1787571288.90 total 40.13 last 14.40)
 - mkdir 681 (time 1787571298.96 total 50.19 last 15.00)
 - mkdir 827 (time 1787571309.02 total 60.25 last 14.52)
 - mkdir 964 (time 1787571319.07 total 70.30 last 13.63)
total: 1000 mkdir in 75.47 seconds: 13.25 ops/second
 - mkdir 140 (time 1787571258.20 total 10.08 last 13.89)
 - mkdir 250 (time 1787571268.36 total 20.24 last 10.82)
 - mkdir 381 (time 1787571278.40 total 30.28 last 13.06)
 - mkdir 525 (time 1787571288.48 total 40.36 last 14.28)
 - mkdir 675 (time 1787571298.52 total 50.40 last 14.93)
 - mkdir 819 (time 1787571308.58 total 60.46 last 14.31)
 - mkdir 956 (time 1787571318.60 total 70.48 last 13.68)
total: 1000 mkdir in 76.44 seconds: 13.08 ops/second
 - mkdir 139 (time 1787571258.10 total 10.05 last 13.83)
 - mkdir 246 (time 1787571268.23 total 20.18 last 10.57)
 - mkdir 368 (time 1787571278.27 total 30.22 last 12.15)
 - mkdir 504 (time 1787571288.35 total 40.30 last 13.50)
 - mkdir 645 (time 1787571298.43 total 50.37 last 13.99)
 - mkdir 781 (time 1787571308.46 total 60.41 last 13.55)
 - mkdir 915 (time 1787571318.48 total 70.43 last 13.38)
total: 1000 mkdir in 79.98 seconds: 12.50 ops/second
Time for file creation 90 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 13 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.40 seconds: 24.76 ops/second
 - mkdir 180 (time 1787571884.41 total 10.01 last 17.98)
 - mkdir 352 (time 1787571894.41 total 20.01 last 17.20)
 - mkdir 521 (time 1787571904.44 total 30.03 last 16.86)
 - mkdir 676 (time 1787571914.48 total 40.08 last 15.43)
 - mkdir 812 (time 1787571924.58 total 50.18 last 13.46)
 - mkdir 964 (time 1787571934.61 total 60.21 last 15.16)
total: 1000 mkdir in 62.98 seconds: 15.88 ops/second
 - mkdir 182 (time 1787571884.23 total 10.02 last 18.17)
 - mkdir 344 (time 1787571894.24 total 20.03 last 16.18)
 - mkdir 510 (time 1787571904.25 total 30.04 last 16.58)
 - mkdir 661 (time 1787571914.27 total 40.05 last 15.08)
 - mkdir 796 (time 1787571924.33 total 50.12 last 13.41)
 - mkdir 939 (time 1787571934.33 total 60.12 last 14.29)
total: 1000 mkdir in 65.59 seconds: 15.25 ops/second
 - mkdir 175 (time 1787571883.07 total 10.03 last 17.45)
 - mkdir 334 (time 1787571893.07 total 20.04 last 15.89)
 - mkdir 507 (time 1787571903.11 total 30.08 last 17.23)
 - mkdir 649 (time 1787571913.16 total 40.12 last 14.14)
 - mkdir 787 (time 1787571923.23 total 50.19 last 13.70)
 - mkdir 925 (time 1787571933.30 total 60.26 last 13.70)
total: 1000 mkdir in 67.07 seconds: 14.91 ops/second
 - mkdir 181 (time 1787571883.57 total 10.03 last 18.05)
 - mkdir 337 (time 1787571893.64 total 20.09 last 15.50)
 - mkdir 508 (time 1787571903.72 total 30.17 last 16.96)
 - mkdir 656 (time 1787571913.72 total 40.18 last 14.80)
 - mkdir 787 (time 1787571923.79 total 50.24 last 13.02)
 - mkdir 929 (time 1787571933.79 total 60.24 last 14.20)
total: 1000 mkdir in 66.68 seconds: 15.00 ops/second
 - mkdir 178 (time 1787571884.45 total 10.02 last 17.76)
 - mkdir 346 (time 1787571894.46 total 20.04 last 16.78)
 - mkdir 507 (time 1787571904.51 total 30.09 last 16.02)
 - mkdir 659 (time 1787571914.52 total 40.09 last 15.19)
 - mkdir 787 (time 1787571924.60 total 50.18 last 12.69)
 - mkdir 934 (time 1787571934.63 total 60.21 last 14.66)
total: 1000 mkdir in 66.57 seconds: 15.02 ops/second
 - mkdir 181 (time 1787571884.36 total 10.07 last 17.98)
 - mkdir 347 (time 1787571894.36 total 20.07 last 16.60)
 - mkdir 512 (time 1787571904.44 total 30.14 last 16.38)
 - mkdir 659 (time 1787571914.47 total 40.18 last 14.65)
 - mkdir 785 (time 1787571924.49 total 50.19 last 12.58)
 - mkdir 928 (time 1787571934.51 total 60.22 last 14.26)
total: 1000 mkdir in 67.21 seconds: 14.88 ops/second
 - mkdir 175 (time 1787571883.65 total 10.04 last 17.43)
 - mkdir 334 (time 1787571893.67 total 20.06 last 15.87)
 - mkdir 497 (time 1787571903.68 total 30.06 last 16.30)
 - mkdir 641 (time 1787571913.69 total 40.08 last 14.38)
 - mkdir 775 (time 1787571923.79 total 50.18 last 13.27)
 - mkdir 916 (time 1787571933.84 total 60.22 last 14.03)
total: 1000 mkdir in 68.08 seconds: 14.69 ops/second
 - mkdir 177 (time 1787571884.42 total 10.02 last 17.67)
 - mkdir 342 (time 1787571894.43 total 20.03 last 16.48)
 - mkdir 498 (time 1787571904.44 total 30.03 last 15.59)
 - mkdir 646 (time 1787571914.47 total 40.07 last 14.75)
 - mkdir 776 (time 1787571924.50 total 50.10 last 12.96)
 - mkdir 918 (time 1787571934.51 total 60.11 last 14.19)
total: 1000 mkdir in 68.30 seconds: 14.64 ops/second
 - mkdir 175 (time 1787571884.54 total 10.01 last 17.48)
 - mkdir 342 (time 1787571894.56 total 20.04 last 16.66)
 - mkdir 501 (time 1787571904.66 total 30.13 last 15.75)
 - mkdir 649 (time 1787571914.71 total 40.18 last 14.73)
 - mkdir 771 (time 1787571924.72 total 50.19 last 12.19)
 - mkdir 915 (time 1787571934.78 total 60.25 last 14.32)
total: 1000 mkdir in 68.89 seconds: 14.52 ops/second
 - mkdir 164 (time 1787571882.69 total 10.05 last 16.33)
 - mkdir 317 (time 1787571892.70 total 20.05 last 15.29)
 - mkdir 480 (time 1787571902.72 total 30.07 last 16.27)
 - mkdir 611 (time 1787571912.74 total 40.10 last 13.06)
 - mkdir 749 (time 1787571922.90 total 50.25 last 13.58)
 - mkdir 881 (time 1787571932.95 total 60.31 last 13.13)
 - mkdir 983 (time 1787571943.09 total 70.44 last 10.07)
total: 1000 mkdir in 71.62 seconds: 13.96 ops/second
Time for file creation 80 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787571953 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1787571953 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1787571954 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1787571954 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787571954 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787571954 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787571954 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787571954 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787571954 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1787571954 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
Time for file removal 49 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8