== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 05:47:54 (1786960074)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.57 seconds: 17.47 ops/second
 - mkdir 130 (time 1786960098.40 total 10.05 last 12.93)
 - mkdir 269 (time 1786960108.44 total 20.09 last 13.85)
 - mkdir 418 (time 1786960118.46 total 30.11 last 14.87)
 - mkdir 571 (time 1786960128.51 total 40.17 last 15.22)
 - mkdir 729 (time 1786960138.57 total 50.22 last 15.72)
 - mkdir 867 (time 1786960148.60 total 60.25 last 13.76)
total: 1000 mkdir in 68.69 seconds: 14.56 ops/second
 - mkdir 131 (time 1786960097.15 total 10.04 last 13.04)
 - mkdir 271 (time 1786960107.23 total 20.12 last 13.90)
 - mkdir 412 (time 1786960117.23 total 30.12 last 14.10)
 - mkdir 557 (time 1786960127.24 total 40.13 last 14.49)
 - mkdir 710 (time 1786960137.27 total 50.16 last 15.25)
 - mkdir 846 (time 1786960147.32 total 60.21 last 13.54)
total: 1000 mkdir in 70.16 seconds: 14.25 ops/second
 - mkdir 133 (time 1786960098.72 total 10.05 last 13.23)
 - mkdir 265 (time 1786960108.77 total 20.10 last 13.14)
 - mkdir 414 (time 1786960118.78 total 30.11 last 14.88)
 - mkdir 566 (time 1786960128.84 total 40.17 last 15.11)
 - mkdir 725 (time 1786960138.89 total 50.22 last 15.82)
 - mkdir 865 (time 1786960148.97 total 60.30 last 13.89)
total: 1000 mkdir in 69.20 seconds: 14.45 ops/second
 - mkdir 140 (time 1786960098.87 total 10.01 last 13.98)
 - mkdir 275 (time 1786960108.87 total 20.02 last 13.49)
 - mkdir 424 (time 1786960118.93 total 30.08 last 14.81)
 - mkdir 575 (time 1786960128.96 total 40.11 last 15.06)
 - mkdir 728 (time 1786960138.99 total 50.13 last 15.26)
 - mkdir 866 (time 1786960149.00 total 60.14 last 13.79)
total: 1000 mkdir in 69.07 seconds: 14.48 ops/second
 - mkdir 136 (time 1786960098.92 total 10.02 last 13.57)
 - mkdir 271 (time 1786960108.94 total 20.05 last 13.46)
 - mkdir 420 (time 1786960118.97 total 30.08 last 14.86)
 - mkdir 574 (time 1786960129.02 total 40.12 last 15.33)
 - mkdir 731 (time 1786960139.05 total 50.15 last 15.66)
 - mkdir 868 (time 1786960149.05 total 60.16 last 13.69)
total: 1000 mkdir in 69.22 seconds: 14.45 ops/second
 - mkdir 137 (time 1786960099.04 total 10.05 last 13.63)
 - mkdir 271 (time 1786960109.11 total 20.12 last 13.30)
 - mkdir 420 (time 1786960119.17 total 30.19 last 14.81)
 - mkdir 572 (time 1786960129.19 total 40.20 last 15.18)
 - mkdir 728 (time 1786960139.22 total 50.23 last 15.55)
 - mkdir 865 (time 1786960149.26 total 60.27 last 13.65)
total: 1000 mkdir in 69.38 seconds: 14.41 ops/second
 - mkdir 134 (time 1786960098.49 total 10.03 last 13.36)
 - mkdir 259 (time 1786960108.56 total 20.10 last 12.42)
 - mkdir 403 (time 1786960118.58 total 30.11 last 14.37)
 - mkdir 556 (time 1786960128.63 total 40.17 last 15.22)
 - mkdir 712 (time 1786960138.66 total 50.20 last 15.54)
 - mkdir 848 (time 1786960148.72 total 60.26 last 13.52)
 - mkdir 986 (time 1786960158.73 total 70.27 last 13.79)
total: 1000 mkdir in 71.72 seconds: 13.94 ops/second
 - mkdir 127 (time 1786960098.19 total 10.06 last 12.63)
 - mkdir 262 (time 1786960108.24 total 20.11 last 13.43)
 - mkdir 411 (time 1786960118.27 total 30.14 last 14.85)
 - mkdir 557 (time 1786960128.28 total 40.15 last 14.59)
 - mkdir 710 (time 1786960138.32 total 50.18 last 15.24)
 - mkdir 842 (time 1786960148.35 total 60.21 last 13.16)
 - mkdir 985 (time 1786960158.38 total 70.25 last 14.25)
total: 1000 mkdir in 72.04 seconds: 13.88 ops/second
 - mkdir 127 (time 1786960098.46 total 10.00 last 12.70)
 - mkdir 253 (time 1786960108.54 total 20.08 last 12.50)
 - mkdir 398 (time 1786960118.57 total 30.12 last 14.45)
 - mkdir 550 (time 1786960128.58 total 40.12 last 15.19)
 - mkdir 706 (time 1786960138.61 total 50.16 last 15.55)
 - mkdir 840 (time 1786960148.63 total 60.17 last 13.38)
 - mkdir 983 (time 1786960158.73 total 70.27 last 14.16)
total: 1000 mkdir in 72.14 seconds: 13.86 ops/second
 - mkdir 132 (time 1786960099.22 total 10.04 last 13.14)
 - mkdir 257 (time 1786960109.29 total 20.11 last 12.42)
 - mkdir 398 (time 1786960119.29 total 30.11 last 14.10)
 - mkdir 545 (time 1786960129.30 total 40.12 last 14.70)
 - mkdir 700 (time 1786960139.32 total 50.14 last 15.46)
 - mkdir 837 (time 1786960149.32 total 60.14 last 13.70)
 - mkdir 972 (time 1786960159.44 total 70.26 last 13.35)
total: 1000 mkdir in 72.72 seconds: 13.75 ops/second
Time for file creation 87 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 18 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.58 seconds: 17.30 ops/second
 - mkdir 193 (time 1786960732.36 total 10.05 last 19.21)
 - mkdir 356 (time 1786960742.40 total 20.08 last 16.25)
 - mkdir 511 (time 1786960752.50 total 30.18 last 15.35)
 - mkdir 645 (time 1786960762.56 total 40.24 last 13.32)
 - mkdir 818 (time 1786960772.61 total 50.29 last 17.21)
 - mkdir 940 (time 1786960782.62 total 60.31 last 12.18)
total: 1000 mkdir in 65.39 seconds: 15.29 ops/second
 - mkdir 178 (time 1786960731.34 total 10.01 last 17.79)
 - mkdir 340 (time 1786960741.35 total 20.02 last 16.18)
 - mkdir 489 (time 1786960751.41 total 30.08 last 14.81)
 - mkdir 624 (time 1786960761.47 total 40.14 last 13.42)
 - mkdir 786 (time 1786960771.58 total 50.25 last 16.03)
 - mkdir 910 (time 1786960781.65 total 60.32 last 12.32)
 - mkdir 996 (time 1786960791.78 total 70.45 last 8.49)
total: 1000 mkdir in 71.14 seconds: 14.06 ops/second
 - mkdir 194 (time 1786960732.48 total 10.04 last 19.33)
 - mkdir 354 (time 1786960742.52 total 20.07 last 15.94)
 - mkdir 503 (time 1786960752.56 total 30.12 last 14.83)
 - mkdir 634 (time 1786960762.61 total 40.16 last 13.05)
 - mkdir 797 (time 1786960772.62 total 50.18 last 16.27)
 - mkdir 919 (time 1786960782.69 total 60.24 last 12.12)
total: 1000 mkdir in 70.03 seconds: 14.28 ops/second
 - mkdir 185 (time 1786960731.66 total 10.04 last 18.43)
 - mkdir 345 (time 1786960741.68 total 20.06 last 15.97)
 - mkdir 492 (time 1786960751.75 total 30.13 last 14.60)
 - mkdir 628 (time 1786960761.85 total 40.23 last 13.46)
 - mkdir 793 (time 1786960771.87 total 50.24 last 16.47)
 - mkdir 915 (time 1786960781.99 total 60.37 last 12.05)
 - mkdir 997 (time 1786960792.13 total 70.51 last 8.09)
total: 1000 mkdir in 70.93 seconds: 14.10 ops/second
 - mkdir 194 (time 1786960733.06 total 10.01 last 19.38)
 - mkdir 353 (time 1786960743.11 total 20.05 last 15.83)
 - mkdir 497 (time 1786960753.13 total 30.08 last 14.36)
 - mkdir 632 (time 1786960763.14 total 40.09 last 13.49)
 - mkdir 791 (time 1786960773.18 total 50.13 last 15.83)
 - mkdir 913 (time 1786960783.22 total 60.17 last 12.16)
 - mkdir 996 (time 1786960793.27 total 70.22 last 8.26)
total: 1000 mkdir in 70.80 seconds: 14.12 ops/second
 - mkdir 182 (time 1786960732.59 total 10.01 last 18.19)
 - mkdir 343 (time 1786960742.64 total 20.05 last 16.03)
 - mkdir 492 (time 1786960752.67 total 30.09 last 14.85)
 - mkdir 625 (time 1786960762.68 total 40.10 last 13.28)
 - mkdir 788 (time 1786960772.72 total 50.14 last 16.24)
 - mkdir 914 (time 1786960782.84 total 60.25 last 12.46)
 - mkdir 995 (time 1786960793.03 total 70.44 last 7.95)
total: 1000 mkdir in 71.28 seconds: 14.03 ops/second
 - mkdir 185 (time 1786960733.03 total 10.04 last 18.42)
 - mkdir 341 (time 1786960743.09 total 20.10 last 15.51)
 - mkdir 482 (time 1786960753.12 total 30.13 last 14.05)
 - mkdir 619 (time 1786960763.17 total 40.18 last 13.64)
 - mkdir 777 (time 1786960773.18 total 50.19 last 15.77)
 - mkdir 903 (time 1786960783.19 total 60.20 last 12.59)
 - mkdir 985 (time 1786960793.27 total 70.28 last 8.14)
total: 1000 mkdir in 72.80 seconds: 13.74 ops/second
 - mkdir 188 (time 1786960733.38 total 10.08 last 18.65)
 - mkdir 342 (time 1786960743.42 total 20.12 last 15.34)
 - mkdir 486 (time 1786960753.49 total 30.20 last 14.29)
 - mkdir 624 (time 1786960763.51 total 40.22 last 13.77)
 - mkdir 780 (time 1786960773.54 total 50.24 last 15.56)
 - mkdir 898 (time 1786960783.57 total 60.27 last 11.77)
 - mkdir 974 (time 1786960793.67 total 70.37 last 7.52)
total: 1000 mkdir in 74.70 seconds: 13.39 ops/second
 - mkdir 182 (time 1786960732.92 total 10.05 last 18.11)
 - mkdir 339 (time 1786960743.00 total 20.13 last 15.57)
 - mkdir 480 (time 1786960753.11 total 30.24 last 13.95)
 - mkdir 611 (time 1786960763.13 total 40.26 last 13.07)
 - mkdir 765 (time 1786960773.14 total 50.27 last 15.39)
 - mkdir 884 (time 1786960783.19 total 60.32 last 11.84)
 - mkdir 961 (time 1786960793.28 total 70.41 last 7.63)
total: 1000 mkdir in 76.30 seconds: 13.11 ops/second
 - mkdir 179 (time 1786960732.78 total 10.03 last 17.85)
 - mkdir 331 (time 1786960742.82 total 20.07 last 15.13)
 - mkdir 469 (time 1786960752.86 total 30.11 last 13.75)
 - mkdir 597 (time 1786960762.92 total 40.17 last 12.72)
 - mkdir 749 (time 1786960772.98 total 50.23 last 15.12)
 - mkdir 865 (time 1786960783.00 total 60.25 last 11.57)
 - mkdir 936 (time 1786960793.01 total 70.26 last 7.09)
total: 1000 mkdir in 78.56 seconds: 12.73 ops/second
Time for file creation 90 sec for 1 RPC sent at a time
 - unlinked 0 (time 1786960812 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1786960812 ; total 1 ; last 1)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1786960812 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1786960813 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1786960812 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1786960813 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1786960813 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1786960813 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1786960813 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1786960813 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
Time for file removal 50 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8