== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 16:31:38 (1790022698)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.48 seconds: 20.97 ops/second
 - mkdir 125 (time 1790022723.15 total 10.04 last 12.45)
 - mkdir 284 (time 1790022733.19 total 20.07 last 15.85)
 - mkdir 438 (time 1790022743.25 total 30.13 last 15.30)
 - mkdir 592 (time 1790022753.27 total 40.15 last 15.37)
 - mkdir 728 (time 1790022763.28 total 50.17 last 13.58)
 - mkdir 865 (time 1790022773.29 total 60.18 last 13.68)
total: 1000 mkdir in 69.99 seconds: 14.29 ops/second
 - mkdir 135 (time 1790022724.76 total 10.04 last 13.44)
 - mkdir 296 (time 1790022734.76 total 20.05 last 16.09)
 - mkdir 446 (time 1790022744.82 total 30.10 last 14.92)
 - mkdir 607 (time 1790022754.89 total 40.18 last 15.98)
 - mkdir 741 (time 1790022764.90 total 50.19 last 13.39)
 - mkdir 877 (time 1790022775.00 total 60.28 last 13.47)
total: 1000 mkdir in 70.18 seconds: 14.25 ops/second
 - mkdir 123 (time 1790022720.33 total 10.09 last 12.19)
 - mkdir 268 (time 1790022730.34 total 20.11 last 14.48)
 - mkdir 413 (time 1790022740.36 total 30.13 last 14.47)
 - mkdir 552 (time 1790022750.39 total 40.16 last 13.86)
 - mkdir 691 (time 1790022760.43 total 50.19 last 13.86)
 - mkdir 818 (time 1790022770.46 total 60.22 last 12.66)
 - mkdir 953 (time 1790022780.51 total 70.28 last 13.43)
total: 1000 mkdir in 75.16 seconds: 13.31 ops/second
 - mkdir 143 (time 1790022724.26 total 10.03 last 14.26)
 - mkdir 302 (time 1790022734.33 total 20.11 last 15.78)
 - mkdir 446 (time 1790022744.34 total 30.12 last 14.39)
 - mkdir 603 (time 1790022754.34 total 40.12 last 15.69)
 - mkdir 733 (time 1790022764.40 total 50.17 last 12.93)
 - mkdir 870 (time 1790022774.45 total 60.23 last 13.62)
 - mkdir 988 (time 1790022784.55 total 70.32 last 11.69)
total: 1000 mkdir in 72.26 seconds: 13.84 ops/second
 - mkdir 121 (time 1790022723.55 total 10.02 last 12.08)
 - mkdir 278 (time 1790022733.57 total 20.03 last 15.68)
 - mkdir 430 (time 1790022743.59 total 30.05 last 15.16)
 - mkdir 589 (time 1790022753.60 total 40.07 last 15.88)
 - mkdir 725 (time 1790022763.64 total 50.10 last 13.56)
 - mkdir 855 (time 1790022773.69 total 60.15 last 12.93)
 - mkdir 982 (time 1790022783.84 total 70.30 last 12.51)
total: 1000 mkdir in 73.23 seconds: 13.66 ops/second
 - mkdir 138 (time 1790022724.79 total 10.02 last 13.78)
 - mkdir 298 (time 1790022734.79 total 20.02 last 15.99)
 - mkdir 444 (time 1790022744.83 total 30.05 last 14.55)
 - mkdir 603 (time 1790022754.95 total 40.17 last 15.71)
 - mkdir 739 (time 1790022764.95 total 50.18 last 13.59)
 - mkdir 873 (time 1790022774.97 total 60.20 last 13.38)
 - mkdir 989 (time 1790022785.02 total 70.25 last 11.54)
total: 1000 mkdir in 72.05 seconds: 13.88 ops/second
 - mkdir 135 (time 1790022724.58 total 10.06 last 13.42)
 - mkdir 294 (time 1790022734.58 total 20.06 last 15.90)
 - mkdir 441 (time 1790022744.67 total 30.15 last 14.57)
 - mkdir 597 (time 1790022754.68 total 40.16 last 15.57)
 - mkdir 729 (time 1790022764.69 total 50.17 last 13.19)
 - mkdir 864 (time 1790022774.71 total 60.19 last 13.48)
 - mkdir 985 (time 1790022784.82 total 70.30 last 11.97)
total: 1000 mkdir in 72.66 seconds: 13.76 ops/second
 - mkdir 136 (time 1790022724.77 total 10.03 last 13.56)
 - mkdir 289 (time 1790022734.79 total 20.05 last 15.27)
 - mkdir 429 (time 1790022744.85 total 30.11 last 13.92)
 - mkdir 589 (time 1790022754.90 total 40.17 last 15.91)
 - mkdir 723 (time 1790022764.97 total 50.23 last 13.31)
 - mkdir 860 (time 1790022775.00 total 60.27 last 13.65)
 - mkdir 979 (time 1790022785.08 total 70.34 last 11.81)
total: 1000 mkdir in 73.59 seconds: 13.59 ops/second
 - mkdir 134 (time 1790022725.27 total 10.03 last 13.36)
 - mkdir 288 (time 1790022735.33 total 20.09 last 15.31)
 - mkdir 432 (time 1790022745.37 total 30.13 last 14.34)
 - mkdir 588 (time 1790022755.41 total 40.17 last 15.55)
 - mkdir 717 (time 1790022765.45 total 50.21 last 12.85)
 - mkdir 853 (time 1790022775.51 total 60.27 last 13.52)
 - mkdir 965 (time 1790022785.62 total 70.38 last 11.08)
total: 1000 mkdir in 74.92 seconds: 13.35 ops/second
 - mkdir 133 (time 1790022724.17 total 10.09 last 13.18)
 - mkdir 284 (time 1790022734.24 total 20.15 last 15.00)
 - mkdir 429 (time 1790022744.26 total 30.18 last 14.46)
 - mkdir 582 (time 1790022754.28 total 40.20 last 15.28)
 - mkdir 705 (time 1790022764.46 total 50.38 last 12.08)
 - mkdir 835 (time 1790022774.47 total 60.39 last 12.99)
 - mkdir 955 (time 1790022784.60 total 70.52 last 11.85)
total: 1000 mkdir in 76.16 seconds: 13.13 ops/second
Time for file creation 90 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 14 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.43 seconds: 23.16 ops/second
 - mkdir 153 (time 1790023365.39 total 10.05 last 15.22)
 - mkdir 316 (time 1790023375.42 total 20.08 last 16.26)
 - mkdir 459 (time 1790023385.45 total 30.11 last 14.25)
 - mkdir 608 (time 1790023395.46 total 40.11 last 14.89)
 - mkdir 767 (time 1790023405.50 total 50.16 last 15.84)
 - mkdir 942 (time 1790023415.50 total 60.16 last 17.49)
total: 1000 mkdir in 64.07 seconds: 15.61 ops/second
 - mkdir 169 (time 1790023366.78 total 10.07 last 16.78)
 - mkdir 331 (time 1790023376.81 total 20.10 last 16.15)
 - mkdir 477 (time 1790023386.82 total 30.11 last 14.59)
 - mkdir 628 (time 1790023396.87 total 40.16 last 15.03)
 - mkdir 789 (time 1790023406.93 total 50.22 last 16.00)
 - mkdir 962 (time 1790023417.03 total 60.33 last 17.12)
total: 1000 mkdir in 62.73 seconds: 15.94 ops/second
 - mkdir 148 (time 1790023364.84 total 10.01 last 14.78)
 - mkdir 313 (time 1790023374.95 total 20.12 last 16.32)
 - mkdir 453 (time 1790023384.99 total 30.17 last 13.94)
 - mkdir 607 (time 1790023395.00 total 40.17 last 15.39)
 - mkdir 765 (time 1790023405.03 total 50.21 last 15.75)
 - mkdir 933 (time 1790023415.05 total 60.23 last 16.77)
total: 1000 mkdir in 64.79 seconds: 15.44 ops/second
 - mkdir 148 (time 1790023364.49 total 10.10 last 14.65)
 - mkdir 313 (time 1790023374.53 total 20.13 last 16.45)
 - mkdir 445 (time 1790023384.55 total 30.15 last 13.17)
 - mkdir 592 (time 1790023394.55 total 40.16 last 14.69)
 - mkdir 746 (time 1790023404.63 total 50.24 last 15.28)
 - mkdir 921 (time 1790023414.70 total 60.30 last 17.39)
total: 1000 mkdir in 65.79 seconds: 15.20 ops/second
 - mkdir 160 (time 1790023366.25 total 10.00 last 16.00)
 - mkdir 315 (time 1790023376.29 total 20.04 last 15.45)
 - mkdir 459 (time 1790023386.34 total 30.09 last 14.32)
 - mkdir 609 (time 1790023396.36 total 40.11 last 14.98)
 - mkdir 769 (time 1790023406.39 total 50.14 last 15.94)
 - mkdir 937 (time 1790023416.42 total 60.17 last 16.76)
total: 1000 mkdir in 65.22 seconds: 15.33 ops/second
 - mkdir 156 (time 1790023365.99 total 10.01 last 15.59)
 - mkdir 307 (time 1790023376.02 total 20.03 last 15.06)
 - mkdir 451 (time 1790023386.02 total 30.04 last 14.39)
 - mkdir 601 (time 1790023396.03 total 40.05 last 14.99)
 - mkdir 759 (time 1790023406.04 total 50.06 last 15.78)
 - mkdir 926 (time 1790023416.05 total 60.06 last 16.69)
total: 1000 mkdir in 65.79 seconds: 15.20 ops/second
 - mkdir 162 (time 1790023366.71 total 10.01 last 16.18)
 - mkdir 316 (time 1790023376.71 total 20.02 last 15.39)
 - mkdir 455 (time 1790023386.76 total 30.07 last 13.83)
 - mkdir 607 (time 1790023396.81 total 40.12 last 15.12)
 - mkdir 762 (time 1790023406.88 total 50.19 last 15.40)
 - mkdir 936 (time 1790023416.90 total 60.21 last 17.36)
total: 1000 mkdir in 65.21 seconds: 15.34 ops/second
 - mkdir 161 (time 1790023366.87 total 10.12 last 15.91)
 - mkdir 318 (time 1790023376.94 total 20.18 last 15.60)
 - mkdir 458 (time 1790023387.00 total 30.24 last 13.92)
 - mkdir 611 (time 1790023397.01 total 40.25 last 15.29)
 - mkdir 763 (time 1790023407.04 total 50.28 last 15.15)
 - mkdir 927 (time 1790023417.09 total 60.33 last 16.32)
total: 1000 mkdir in 66.64 seconds: 15.01 ops/second
 - mkdir 158 (time 1790023366.81 total 10.06 last 15.70)
 - mkdir 312 (time 1790023376.88 total 20.13 last 15.29)
 - mkdir 451 (time 1790023386.91 total 30.17 last 13.85)
 - mkdir 597 (time 1790023396.94 total 40.20 last 14.55)
 - mkdir 743 (time 1790023406.97 total 50.23 last 14.56)
 - mkdir 908 (time 1790023417.04 total 60.30 last 16.39)
total: 1000 mkdir in 68.15 seconds: 14.67 ops/second
 - mkdir 139 (time 1790023365.01 total 10.06 last 13.82)
 - mkdir 296 (time 1790023375.09 total 20.14 last 15.57)
 - mkdir 425 (time 1790023385.15 total 30.19 last 12.83)
 - mkdir 563 (time 1790023395.16 total 40.21 last 13.78)
 - mkdir 716 (time 1790023405.22 total 50.27 last 15.21)
 - mkdir 881 (time 1790023415.26 total 60.31 last 16.44)
 - mkdir 997 (time 1790023425.34 total 70.39 last 11.51)
total: 1000 mkdir in 70.49 seconds: 14.19 ops/second
Time for file creation 81 sec for 1 RPC sent at a time
 - unlinked 0 (time 1790023433 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1790023434 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1790023436 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1790023437 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1790023437 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1790023436 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1790023437 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1790023437 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1790023436 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1790023437 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
Time for file removal 45 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8