== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 14:06:56 (1786730816)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.63 seconds: 15.94 ops/second
 - mkdir 116 (time 1786730836.94 total 10.02 last 11.57)
 - mkdir 238 (time 1786730846.97 total 20.05 last 12.17)
 - mkdir 373 (time 1786730857.00 total 30.08 last 13.46)
 - mkdir 547 (time 1786730867.03 total 40.11 last 17.34)
 - mkdir 662 (time 1786730877.06 total 50.14 last 11.47)
 - mkdir 791 (time 1786730887.08 total 60.16 last 12.87)
 - mkdir 884 (time 1786730897.13 total 70.21 last 9.25)
 - mkdir 976 (time 1786730907.21 total 80.29 last 9.12)
total: 1000 mkdir in 82.79 seconds: 12.08 ops/second
 - mkdir 116 (time 1786730837.91 total 10.12 last 11.46)
 - mkdir 234 (time 1786730847.97 total 20.18 last 11.73)
 - mkdir 376 (time 1786730858.04 total 30.25 last 14.10)
 - mkdir 543 (time 1786730868.09 total 40.30 last 16.63)
 - mkdir 660 (time 1786730878.09 total 50.30 last 11.70)
 - mkdir 791 (time 1786730888.13 total 60.34 last 13.05)
 - mkdir 888 (time 1786730898.14 total 70.35 last 9.69)
 - mkdir 977 (time 1786730908.17 total 80.38 last 8.87)
total: 1000 mkdir in 82.89 seconds: 12.06 ops/second
 - mkdir 103 (time 1786730837.88 total 10.01 last 10.29)
 - mkdir 221 (time 1786730847.94 total 20.07 last 11.73)
 - mkdir 365 (time 1786730857.98 total 30.11 last 14.34)
 - mkdir 530 (time 1786730868.02 total 40.15 last 16.43)
 - mkdir 652 (time 1786730878.04 total 50.18 last 12.17)
 - mkdir 783 (time 1786730888.12 total 60.26 last 12.99)
 - mkdir 882 (time 1786730898.16 total 70.29 last 9.86)
 - mkdir 972 (time 1786730908.26 total 80.39 last 8.91)
total: 1000 mkdir in 83.78 seconds: 11.94 ops/second
 - mkdir 111 (time 1786730838.34 total 10.03 last 11.06)
 - mkdir 237 (time 1786730848.36 total 20.06 last 12.57)
 - mkdir 380 (time 1786730858.42 total 30.11 last 14.22)
 - mkdir 543 (time 1786730868.43 total 40.13 last 16.27)
 - mkdir 664 (time 1786730878.48 total 50.18 last 12.05)
 - mkdir 788 (time 1786730888.49 total 60.19 last 12.38)
 - mkdir 890 (time 1786730898.53 total 70.22 last 10.17)
 - mkdir 976 (time 1786730908.57 total 80.26 last 8.57)
total: 1000 mkdir in 83.73 seconds: 11.94 ops/second
 - mkdir 111 (time 1786730838.64 total 10.04 last 11.06)
 - mkdir 233 (time 1786730848.67 total 20.07 last 12.16)
 - mkdir 377 (time 1786730858.74 total 30.14 last 14.30)
 - mkdir 548 (time 1786730868.78 total 40.18 last 17.03)
 - mkdir 665 (time 1786730878.79 total 50.20 last 11.68)
 - mkdir 791 (time 1786730888.86 total 60.26 last 12.52)
 - mkdir 888 (time 1786730898.92 total 70.32 last 9.64)
 - mkdir 973 (time 1786730908.98 total 80.39 last 8.45)
total: 1000 mkdir in 84.66 seconds: 11.81 ops/second
 - mkdir 113 (time 1786730838.35 total 10.12 last 11.16)
 - mkdir 229 (time 1786730848.36 total 20.12 last 11.60)
 - mkdir 375 (time 1786730858.44 total 30.20 last 14.48)
 - mkdir 543 (time 1786730868.50 total 40.27 last 16.70)
 - mkdir 659 (time 1786730878.51 total 50.28 last 11.58)
 - mkdir 785 (time 1786730888.52 total 60.28 last 12.59)
 - mkdir 885 (time 1786730898.64 total 70.41 last 9.88)
 - mkdir 970 (time 1786730908.68 total 80.45 last 8.47)
total: 1000 mkdir in 85.15 seconds: 11.74 ops/second
 - mkdir 106 (time 1786730838.50 total 10.11 last 10.48)
 - mkdir 226 (time 1786730848.56 total 20.17 last 11.93)
 - mkdir 366 (time 1786730858.59 total 30.20 last 13.96)
 - mkdir 532 (time 1786730868.61 total 40.21 last 16.57)
 - mkdir 652 (time 1786730878.69 total 50.29 last 11.90)
 - mkdir 779 (time 1786730888.70 total 60.31 last 12.68)
 - mkdir 877 (time 1786730898.79 total 70.39 last 9.72)
 - mkdir 962 (time 1786730908.84 total 80.45 last 8.45)
total: 1000 mkdir in 86.79 seconds: 11.52 ops/second
 - mkdir 114 (time 1786730838.45 total 10.09 last 11.30)
 - mkdir 230 (time 1786730848.46 total 20.10 last 11.59)
 - mkdir 369 (time 1786730858.48 total 30.12 last 13.87)
 - mkdir 531 (time 1786730868.50 total 40.14 last 16.17)
 - mkdir 646 (time 1786730878.51 total 50.15 last 11.49)
 - mkdir 766 (time 1786730888.55 total 60.19 last 11.96)
 - mkdir 869 (time 1786730898.67 total 70.31 last 10.18)
 - mkdir 955 (time 1786730908.73 total 80.37 last 8.55)
total: 1000 mkdir in 88.49 seconds: 11.30 ops/second
 - mkdir 105 (time 1786730837.49 total 10.10 last 10.40)
 - mkdir 222 (time 1786730847.57 total 20.18 last 11.61)
 - mkdir 355 (time 1786730857.58 total 30.19 last 13.28)
 - mkdir 520 (time 1786730867.65 total 40.26 last 16.39)
 - mkdir 633 (time 1786730877.66 total 50.27 last 11.29)
 - mkdir 761 (time 1786730887.69 total 60.29 last 12.76)
 - mkdir 855 (time 1786730897.74 total 70.35 last 9.35)
 - mkdir 942 (time 1786730907.80 total 80.41 last 8.66)
total: 1000 mkdir in 89.95 seconds: 11.12 ops/second
 - mkdir 113 (time 1786730836.15 total 10.02 last 11.28)
 - mkdir 224 (time 1786730846.21 total 20.09 last 11.03)
 - mkdir 338 (time 1786730856.23 total 30.10 last 11.38)
 - mkdir 504 (time 1786730866.26 total 40.13 last 16.55)
 - mkdir 617 (time 1786730876.30 total 50.18 last 11.25)
 - mkdir 748 (time 1786730886.32 total 60.19 last 13.08)
 - mkdir 845 (time 1786730896.32 total 70.20 last 9.70)
 - mkdir 929 (time 1786730906.48 total 80.35 last 8.27)
 - mkdir 989 (time 1786730916.63 total 90.50 last 5.91)
total: 1000 mkdir in 91.95 seconds: 10.88 ops/second
Time for file creation 100 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 12 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.51 seconds: 19.75 ops/second
 - mkdir 129 (time 1786731533.19 total 10.01 last 12.88)
 - mkdir 272 (time 1786731543.23 total 20.04 last 14.26)
 - mkdir 404 (time 1786731553.23 total 30.04 last 13.20)
 - mkdir 558 (time 1786731563.30 total 40.11 last 15.29)
 - mkdir 707 (time 1786731573.32 total 50.13 last 14.87)
 - mkdir 845 (time 1786731583.33 total 60.15 last 13.78)
 - mkdir 992 (time 1786731593.35 total 70.17 last 14.67)
total: 1000 mkdir in 70.61 seconds: 14.16 ops/second
 - mkdir 134 (time 1786731534.10 total 10.05 last 13.33)
 - mkdir 282 (time 1786731544.12 total 20.07 last 14.77)
 - mkdir 424 (time 1786731554.14 total 30.09 last 14.18)
 - mkdir 574 (time 1786731564.16 total 40.11 last 14.97)
 - mkdir 724 (time 1786731574.25 total 50.20 last 14.87)
 - mkdir 857 (time 1786731584.27 total 60.22 last 13.27)
 - mkdir 994 (time 1786731594.39 total 70.34 last 13.54)
total: 1000 mkdir in 70.97 seconds: 14.09 ops/second
 - mkdir 133 (time 1786731534.13 total 10.10 last 13.16)
 - mkdir 277 (time 1786731544.14 total 20.11 last 14.38)
 - mkdir 413 (time 1786731554.16 total 30.13 last 13.58)
 - mkdir 566 (time 1786731564.21 total 40.19 last 15.21)
 - mkdir 715 (time 1786731574.27 total 50.24 last 14.82)
 - mkdir 851 (time 1786731584.32 total 60.30 last 13.53)
 - mkdir 993 (time 1786731594.41 total 70.39 last 14.07)
total: 1000 mkdir in 71.06 seconds: 14.07 ops/second
 - mkdir 134 (time 1786731534.63 total 10.09 last 13.28)
 - mkdir 277 (time 1786731544.67 total 20.13 last 14.25)
 - mkdir 416 (time 1786731554.68 total 30.13 last 13.89)
 - mkdir 570 (time 1786731564.77 total 40.23 last 15.26)
 - mkdir 720 (time 1786731574.81 total 50.27 last 14.94)
 - mkdir 858 (time 1786731584.85 total 60.31 last 13.75)
 - mkdir 996 (time 1786731594.89 total 70.34 last 13.75)
total: 1000 mkdir in 70.91 seconds: 14.10 ops/second
 - mkdir 136 (time 1786731535.04 total 10.00 last 13.60)
 - mkdir 279 (time 1786731545.06 total 20.03 last 14.27)
 - mkdir 421 (time 1786731555.07 total 30.04 last 14.19)
 - mkdir 569 (time 1786731565.10 total 40.06 last 14.76)
 - mkdir 714 (time 1786731575.13 total 50.10 last 14.45)
 - mkdir 853 (time 1786731585.14 total 60.10 last 13.89)
 - mkdir 984 (time 1786731595.21 total 70.18 last 13.00)
total: 1000 mkdir in 71.67 seconds: 13.95 ops/second
 - mkdir 135 (time 1786731534.73 total 10.04 last 13.45)
 - mkdir 271 (time 1786731544.76 total 20.07 last 13.56)
 - mkdir 412 (time 1786731554.82 total 30.12 last 14.02)
 - mkdir 564 (time 1786731564.93 total 40.23 last 15.03)
 - mkdir 713 (time 1786731574.96 total 50.27 last 14.85)
 - mkdir 847 (time 1786731584.97 total 60.27 last 13.39)
 - mkdir 981 (time 1786731595.07 total 70.37 last 13.27)
total: 1000 mkdir in 72.34 seconds: 13.82 ops/second
 - mkdir 131 (time 1786731534.43 total 10.03 last 13.06)
 - mkdir 270 (time 1786731544.49 total 20.09 last 13.82)
 - mkdir 404 (time 1786731554.51 total 30.11 last 13.37)
 - mkdir 560 (time 1786731564.52 total 40.12 last 15.59)
 - mkdir 710 (time 1786731574.59 total 50.19 last 14.90)
 - mkdir 842 (time 1786731584.66 total 60.26 last 13.11)
 - mkdir 980 (time 1786731594.71 total 70.31 last 13.73)
total: 1000 mkdir in 72.64 seconds: 13.77 ops/second
 - mkdir 135 (time 1786731534.81 total 10.00 last 13.50)
 - mkdir 273 (time 1786731544.82 total 20.01 last 13.78)
 - mkdir 409 (time 1786731554.85 total 30.04 last 13.56)
 - mkdir 562 (time 1786731564.92 total 40.12 last 15.19)
 - mkdir 708 (time 1786731574.98 total 50.17 last 14.51)
 - mkdir 847 (time 1786731585.04 total 60.23 last 13.82)
 - mkdir 980 (time 1786731595.09 total 70.29 last 13.23)
total: 1000 mkdir in 72.58 seconds: 13.78 ops/second
 - mkdir 126 (time 1786731534.17 total 10.01 last 12.59)
 - mkdir 266 (time 1786731544.20 total 20.04 last 13.96)
 - mkdir 400 (time 1786731554.23 total 30.06 last 13.37)
 - mkdir 552 (time 1786731564.24 total 40.07 last 15.18)
 - mkdir 704 (time 1786731574.30 total 50.14 last 15.10)
 - mkdir 839 (time 1786731584.32 total 60.15 last 13.48)
 - mkdir 974 (time 1786731594.37 total 70.21 last 13.43)
total: 1000 mkdir in 73.26 seconds: 13.65 ops/second
 - mkdir 121 (time 1786731534.34 total 10.07 last 12.02)
 - mkdir 259 (time 1786731544.45 total 20.17 last 13.65)
 - mkdir 388 (time 1786731554.48 total 30.20 last 12.86)
 - mkdir 542 (time 1786731564.50 total 40.22 last 15.37)
 - mkdir 690 (time 1786731574.54 total 50.26 last 14.74)
 - mkdir 824 (time 1786731584.59 total 60.32 last 13.33)
 - mkdir 955 (time 1786731594.61 total 70.33 last 13.08)
total: 1000 mkdir in 74.79 seconds: 13.37 ops/second
Time for file creation 83 sec for 1 RPC sent at a time
 - unlinked 0 (time 1786731608 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1786731608 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1786731608 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1786731608 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1786731609 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1786731609 ; total 0 ; last 0)
total: 1000 unlinks in 37 seconds: 27.027027 unlinks/second
 - unlinked 0 (time 1786731609 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1786731609 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
 - unlinked 0 (time 1786731609 ; total 1 ; last 1)
total: 1000 unlinks in 39 seconds: 25.641026 unlinks/second
 - unlinked 0 (time 1786731609 ; total 0 ; last 0)
total: 1000 unlinks in 38 seconds: 26.315790 unlinks/second
Time for file removal 52 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8