== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 07:16:04 (1779707764)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.67 seconds: 14.95 ops/second
 - mkdir 111 (time 1779707787.33 total 10.05 last 11.05)
 - mkdir 283 (time 1779707797.37 total 20.09 last 17.13)
 - mkdir 448 (time 1779707807.45 total 30.17 last 16.36)
 - mkdir 587 (time 1779707817.49 total 40.21 last 13.85)
 - mkdir 738 (time 1779707827.50 total 50.22 last 15.09)
 - mkdir 910 (time 1779707837.53 total 60.25 last 17.14)
total: 1000 mkdir in 66.17 seconds: 15.11 ops/second
 - mkdir 115 (time 1779707790.08 total 10.03 last 11.46)
 - mkdir 294 (time 1779707800.11 total 20.07 last 17.85)
 - mkdir 460 (time 1779707810.11 total 30.07 last 16.59)
 - mkdir 599 (time 1779707820.16 total 40.11 last 13.84)
 - mkdir 753 (time 1779707830.16 total 50.12 last 15.39)
 - mkdir 926 (time 1779707840.20 total 60.16 last 17.23)
total: 1000 mkdir in 67.24 seconds: 14.87 ops/second
 - mkdir 109 (time 1779707789.61 total 10.05 last 10.84)
 - mkdir 287 (time 1779707799.66 total 20.11 last 17.71)
 - mkdir 450 (time 1779707809.69 total 30.14 last 16.25)
 - mkdir 591 (time 1779707819.74 total 40.19 last 14.04)
 - mkdir 745 (time 1779707829.78 total 50.23 last 15.33)
 - mkdir 922 (time 1779707839.83 total 60.28 last 17.61)
total: 1000 mkdir in 68.36 seconds: 14.63 ops/second
 - mkdir 107 (time 1779707788.46 total 10.10 last 10.60)
 - mkdir 269 (time 1779707798.49 total 20.12 last 16.16)
 - mkdir 422 (time 1779707808.52 total 30.16 last 15.25)
 - mkdir 562 (time 1779707818.57 total 40.21 last 13.93)
 - mkdir 715 (time 1779707828.62 total 50.25 last 15.23)
 - mkdir 896 (time 1779707838.65 total 60.28 last 18.05)
 - mkdir 994 (time 1779707848.70 total 70.33 last 9.75)
total: 1000 mkdir in 71.17 seconds: 14.05 ops/second
 - mkdir 120 (time 1779707791.18 total 10.05 last 11.94)
 - mkdir 300 (time 1779707801.19 total 20.05 last 17.99)
 - mkdir 455 (time 1779707811.24 total 30.11 last 15.42)
 - mkdir 595 (time 1779707821.27 total 40.14 last 13.96)
 - mkdir 758 (time 1779707831.28 total 50.14 last 16.29)
 - mkdir 929 (time 1779707841.32 total 60.18 last 17.03)
total: 1000 mkdir in 68.72 seconds: 14.55 ops/second
 - mkdir 113 (time 1779707790.38 total 10.01 last 11.29)
 - mkdir 296 (time 1779707800.44 total 20.06 last 18.20)
 - mkdir 452 (time 1779707810.53 total 30.16 last 15.46)
 - mkdir 586 (time 1779707820.54 total 40.17 last 13.38)
 - mkdir 743 (time 1779707830.55 total 50.18 last 15.69)
 - mkdir 919 (time 1779707840.56 total 60.19 last 17.57)
total: 1000 mkdir in 69.49 seconds: 14.39 ops/second
 - mkdir 115 (time 1779707790.67 total 10.03 last 11.47)
 - mkdir 290 (time 1779707800.69 total 20.05 last 17.46)
 - mkdir 450 (time 1779707810.76 total 30.12 last 15.89)
 - mkdir 595 (time 1779707820.80 total 40.16 last 14.45)
 - mkdir 755 (time 1779707830.81 total 50.16 last 15.99)
 - mkdir 923 (time 1779707840.85 total 60.21 last 16.72)
total: 1000 mkdir in 69.61 seconds: 14.37 ops/second
 - mkdir 105 (time 1779707789.93 total 10.03 last 10.46)
 - mkdir 272 (time 1779707799.94 total 20.04 last 16.69)
 - mkdir 432 (time 1779707809.95 total 30.06 last 15.97)
 - mkdir 567 (time 1779707819.98 total 40.08 last 13.46)
 - mkdir 724 (time 1779707830.02 total 50.12 last 15.63)
 - mkdir 902 (time 1779707840.07 total 60.17 last 17.71)
 - mkdir 990 (time 1779707850.09 total 70.19 last 8.78)
total: 1000 mkdir in 71.35 seconds: 14.01 ops/second
 - mkdir 112 (time 1779707790.90 total 10.06 last 11.13)
 - mkdir 291 (time 1779707800.95 total 20.11 last 17.82)
 - mkdir 442 (time 1779707811.07 total 30.23 last 14.92)
 - mkdir 581 (time 1779707821.14 total 40.30 last 13.80)
 - mkdir 744 (time 1779707831.16 total 50.32 last 16.28)
 - mkdir 908 (time 1779707841.16 total 60.32 last 16.39)
 - mkdir 988 (time 1779707851.20 total 70.36 last 7.97)
total: 1000 mkdir in 71.39 seconds: 14.01 ops/second
 - mkdir 114 (time 1779707791.13 total 10.02 last 11.37)
 - mkdir 294 (time 1779707801.15 total 20.04 last 17.97)
 - mkdir 446 (time 1779707811.23 total 30.12 last 15.08)
 - mkdir 585 (time 1779707821.27 total 40.16 last 13.84)
 - mkdir 747 (time 1779707831.29 total 50.18 last 16.17)
 - mkdir 909 (time 1779707841.34 total 60.23 last 16.12)
 - mkdir 990 (time 1779707851.39 total 70.28 last 8.07)
total: 1000 mkdir in 71.18 seconds: 14.05 ops/second
Time for file creation 86 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 16 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.45 seconds: 22.05 ops/second
 - mkdir 132 (time 1779708384.91 total 10.01 last 13.19)
 - mkdir 257 (time 1779708394.94 total 20.04 last 12.46)
 - mkdir 413 (time 1779708404.98 total 30.08 last 15.54)
 - mkdir 560 (time 1779708415.01 total 40.11 last 14.66)
 - mkdir 716 (time 1779708425.04 total 50.14 last 15.55)
 - mkdir 868 (time 1779708435.08 total 60.18 last 15.13)
total: 1000 mkdir in 68.65 seconds: 14.57 ops/second
 - mkdir 128 (time 1779708388.22 total 10.07 last 12.72)
 - mkdir 266 (time 1779708398.28 total 20.12 last 13.72)
 - mkdir 425 (time 1779708408.29 total 30.13 last 15.89)
 - mkdir 571 (time 1779708418.39 total 40.23 last 14.45)
 - mkdir 730 (time 1779708428.43 total 50.27 last 15.84)
 - mkdir 881 (time 1779708438.48 total 60.32 last 15.03)
 - mkdir 993 (time 1779708448.52 total 70.37 last 11.15)
total: 1000 mkdir in 71.44 seconds: 14.00 ops/second
 - mkdir 119 (time 1779708388.66 total 10.02 last 11.88)
 - mkdir 255 (time 1779708398.70 total 20.06 last 13.54)
 - mkdir 405 (time 1779708408.79 total 30.15 last 14.86)
 - mkdir 554 (time 1779708418.83 total 40.19 last 14.84)
 - mkdir 717 (time 1779708428.84 total 50.20 last 16.29)
 - mkdir 873 (time 1779708438.84 total 60.20 last 15.60)
 - mkdir 990 (time 1779708448.85 total 70.21 last 11.68)
total: 1000 mkdir in 71.78 seconds: 13.93 ops/second
 - mkdir 118 (time 1779708389.38 total 10.03 last 11.76)
 - mkdir 263 (time 1779708399.40 total 20.06 last 14.46)
 - mkdir 414 (time 1779708409.48 total 30.13 last 14.99)
 - mkdir 564 (time 1779708419.55 total 40.21 last 14.89)
 - mkdir 730 (time 1779708429.57 total 50.22 last 16.57)
 - mkdir 885 (time 1779708439.61 total 60.26 last 15.44)
 - mkdir 990 (time 1779708449.73 total 70.39 last 10.37)
total: 1000 mkdir in 71.87 seconds: 13.91 ops/second
 - mkdir 122 (time 1779708389.31 total 10.09 last 12.09)
 - mkdir 263 (time 1779708399.36 total 20.14 last 14.02)
 - mkdir 419 (time 1779708409.42 total 30.21 last 15.50)
 - mkdir 569 (time 1779708419.50 total 40.28 last 14.89)
 - mkdir 734 (time 1779708429.52 total 50.30 last 16.47)
 - mkdir 885 (time 1779708439.58 total 60.36 last 15.00)
 - mkdir 988 (time 1779708449.68 total 70.46 last 10.20)
total: 1000 mkdir in 72.21 seconds: 13.85 ops/second
 - mkdir 111 (time 1779708389.16 total 10.14 last 10.95)
 - mkdir 246 (time 1779708399.18 total 20.16 last 13.48)
 - mkdir 400 (time 1779708409.31 total 30.29 last 15.20)
 - mkdir 554 (time 1779708419.35 total 40.33 last 15.34)
 - mkdir 720 (time 1779708429.37 total 50.35 last 16.57)
 - mkdir 872 (time 1779708439.41 total 60.38 last 15.14)
 - mkdir 974 (time 1779708449.61 total 70.59 last 10.00)
total: 1000 mkdir in 74.41 seconds: 13.44 ops/second
 - mkdir 115 (time 1779708389.54 total 10.01 last 11.49)
 - mkdir 252 (time 1779708399.54 total 20.01 last 13.70)
 - mkdir 403 (time 1779708409.68 total 30.15 last 14.89)
 - mkdir 553 (time 1779708419.71 total 40.17 last 14.96)
 - mkdir 717 (time 1779708429.76 total 50.22 last 16.32)
 - mkdir 872 (time 1779708439.76 total 60.23 last 15.49)
 - mkdir 970 (time 1779708449.77 total 70.24 last 9.79)
total: 1000 mkdir in 75.11 seconds: 13.31 ops/second
 - mkdir 116 (time 1779708389.17 total 10.00 last 11.59)
 - mkdir 249 (time 1779708399.19 total 20.03 last 13.27)
 - mkdir 399 (time 1779708409.32 total 30.16 last 14.80)
 - mkdir 545 (time 1779708419.35 total 40.19 last 14.55)
 - mkdir 705 (time 1779708429.38 total 50.21 last 15.97)
 - mkdir 856 (time 1779708439.41 total 60.25 last 15.04)
 - mkdir 963 (time 1779708449.65 total 70.49 last 10.45)
total: 1000 mkdir in 76.04 seconds: 13.15 ops/second
 - mkdir 114 (time 1779708388.97 total 10.02 last 11.38)
 - mkdir 240 (time 1779708398.98 total 20.03 last 12.58)
 - mkdir 389 (time 1779708409.03 total 30.08 last 14.83)
 - mkdir 535 (time 1779708419.09 total 40.14 last 14.51)
 - mkdir 694 (time 1779708429.10 total 50.15 last 15.88)
 - mkdir 846 (time 1779708439.13 total 60.18 last 15.15)
 - mkdir 952 (time 1779708449.34 total 70.39 last 10.39)
total: 1000 mkdir in 77.05 seconds: 12.98 ops/second
 - mkdir 111 (time 1779708389.13 total 10.02 last 11.08)
 - mkdir 246 (time 1779708399.17 total 20.06 last 13.45)
 - mkdir 394 (time 1779708409.32 total 30.21 last 14.59)
 - mkdir 545 (time 1779708419.35 total 40.24 last 15.05)
 - mkdir 703 (time 1779708429.41 total 50.30 last 15.70)
 - mkdir 858 (time 1779708439.44 total 60.33 last 15.46)
 - mkdir 958 (time 1779708449.60 total 70.49 last 9.84)
total: 1000 mkdir in 76.98 seconds: 12.99 ops/second
Time for file creation 90 sec for 1 RPC sent at a time
 - unlinked 0 (time 1779708466 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1779708466 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1779708467 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1779708468 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1779708467 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1779708468 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1779708468 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1779708468 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1779708468 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1779708469 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
Time for file removal 47 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8