== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 09:11:58 (1788527518)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.86 seconds: 11.63 ops/second
 - mkdir 155 (time 1788527538.68 total 10.03 last 15.45)
 - mkdir 291 (time 1788527548.68 total 20.04 last 13.59)
 - mkdir 436 (time 1788527558.70 total 30.06 last 14.48)
 - mkdir 623 (time 1788527568.71 total 40.07 last 18.68)
 - mkdir 811 (time 1788527578.72 total 50.08 last 18.78)
 - mkdir 972 (time 1788527588.79 total 60.15 last 15.99)
total: 1000 mkdir in 61.48 seconds: 16.27 ops/second
 - mkdir 149 (time 1788527540.69 total 10.07 last 14.79)
 - mkdir 297 (time 1788527550.69 total 20.08 last 14.79)
 - mkdir 465 (time 1788527560.72 total 30.11 last 16.76)
 - mkdir 658 (time 1788527570.76 total 40.15 last 19.21)
 - mkdir 843 (time 1788527580.82 total 50.21 last 18.40)
total: 1000 mkdir in 59.59 seconds: 16.78 ops/second
 - mkdir 150 (time 1788527539.15 total 10.09 last 14.86)
 - mkdir 287 (time 1788527549.16 total 20.11 last 13.68)
 - mkdir 433 (time 1788527559.20 total 30.15 last 14.55)
 - mkdir 630 (time 1788527569.25 total 40.20 last 19.60)
 - mkdir 814 (time 1788527579.29 total 50.24 last 18.32)
 - mkdir 980 (time 1788527589.30 total 60.25 last 16.59)
total: 1000 mkdir in 61.38 seconds: 16.29 ops/second
 - mkdir 149 (time 1788527540.36 total 10.07 last 14.80)
 - mkdir 293 (time 1788527550.37 total 20.08 last 14.38)
 - mkdir 450 (time 1788527560.41 total 30.12 last 15.64)
 - mkdir 644 (time 1788527570.42 total 40.13 last 19.38)
 - mkdir 829 (time 1788527580.43 total 50.14 last 18.48)
 - mkdir 992 (time 1788527590.47 total 60.19 last 16.23)
total: 1000 mkdir in 60.64 seconds: 16.49 ops/second
 - mkdir 145 (time 1788527539.73 total 10.04 last 14.44)
 - mkdir 288 (time 1788527549.76 total 20.07 last 14.26)
 - mkdir 446 (time 1788527559.77 total 30.08 last 15.78)
 - mkdir 636 (time 1788527569.79 total 40.11 last 18.95)
 - mkdir 822 (time 1788527579.81 total 50.13 last 18.56)
 - mkdir 981 (time 1788527589.83 total 60.14 last 15.88)
total: 1000 mkdir in 61.29 seconds: 16.32 ops/second
 - mkdir 149 (time 1788527540.69 total 10.09 last 14.76)
 - mkdir 295 (time 1788527550.71 total 20.12 last 14.56)
 - mkdir 455 (time 1788527560.73 total 30.14 last 15.97)
 - mkdir 648 (time 1788527570.78 total 40.18 last 19.22)
 - mkdir 829 (time 1788527580.82 total 50.23 last 18.01)
 - mkdir 992 (time 1788527590.91 total 60.31 last 16.17)
total: 1000 mkdir in 60.84 seconds: 16.44 ops/second
 - mkdir 151 (time 1788527541.48 total 10.02 last 15.06)
 - mkdir 299 (time 1788527551.52 total 20.07 last 14.74)
 - mkdir 463 (time 1788527561.56 total 30.10 last 16.35)
 - mkdir 660 (time 1788527571.62 total 40.16 last 19.58)
 - mkdir 840 (time 1788527581.63 total 50.17 last 17.98)
total: 1000 mkdir in 60.13 seconds: 16.63 ops/second
 - mkdir 153 (time 1788527540.85 total 10.07 last 15.20)
 - mkdir 292 (time 1788527550.86 total 20.08 last 13.88)
 - mkdir 456 (time 1788527560.90 total 30.12 last 16.34)
 - mkdir 646 (time 1788527570.91 total 40.13 last 18.98)
 - mkdir 828 (time 1788527580.98 total 50.20 last 18.08)
 - mkdir 989 (time 1788527590.99 total 60.21 last 16.07)
total: 1000 mkdir in 61.11 seconds: 16.36 ops/second
 - mkdir 148 (time 1788527541.33 total 10.01 last 14.78)
 - mkdir 292 (time 1788527551.34 total 20.02 last 14.39)
 - mkdir 455 (time 1788527561.37 total 30.05 last 16.26)
 - mkdir 644 (time 1788527571.38 total 40.06 last 18.88)
 - mkdir 826 (time 1788527581.44 total 50.12 last 18.10)
 - mkdir 989 (time 1788527591.48 total 60.16 last 16.23)
total: 1000 mkdir in 60.99 seconds: 16.40 ops/second
 - mkdir 142 (time 1788527540.58 total 10.03 last 14.16)
 - mkdir 276 (time 1788527550.58 total 20.03 last 13.40)
 - mkdir 424 (time 1788527560.60 total 30.05 last 14.78)
 - mkdir 610 (time 1788527570.61 total 40.05 last 18.59)
 - mkdir 791 (time 1788527580.70 total 50.15 last 17.93)
 - mkdir 953 (time 1788527590.70 total 60.15 last 16.20)
total: 1000 mkdir in 63.77 seconds: 15.68 ops/second
Time for file creation 73 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 10 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.62 seconds: 16.20 ops/second
 - mkdir 173 (time 1788528116.07 total 10.03 last 17.26)
 - mkdir 325 (time 1788528126.14 total 20.09 last 15.10)
 - mkdir 475 (time 1788528136.16 total 30.11 last 14.98)
 - mkdir 624 (time 1788528146.23 total 40.18 last 14.80)
 - mkdir 781 (time 1788528156.25 total 50.20 last 15.66)
 - mkdir 941 (time 1788528166.33 total 60.29 last 15.87)
total: 1000 mkdir in 63.59 seconds: 15.73 ops/second
 - mkdir 172 (time 1788528116.85 total 10.09 last 17.05)
 - mkdir 328 (time 1788528126.94 total 20.18 last 15.46)
 - mkdir 476 (time 1788528136.94 total 30.18 last 14.79)
 - mkdir 627 (time 1788528146.96 total 40.20 last 15.08)
 - mkdir 778 (time 1788528156.97 total 50.20 last 15.09)
 - mkdir 943 (time 1788528166.99 total 60.23 last 16.46)
total: 1000 mkdir in 63.48 seconds: 15.75 ops/second
 - mkdir 172 (time 1788528117.53 total 10.04 last 17.14)
 - mkdir 328 (time 1788528127.64 total 20.15 last 15.43)
 - mkdir 477 (time 1788528137.73 total 30.24 last 14.77)
 - mkdir 624 (time 1788528147.73 total 40.24 last 14.69)
 - mkdir 775 (time 1788528157.75 total 50.26 last 15.07)
 - mkdir 944 (time 1788528167.76 total 60.27 last 16.88)
total: 1000 mkdir in 64.49 seconds: 15.51 ops/second
 - mkdir 171 (time 1788528117.67 total 10.04 last 17.03)
 - mkdir 320 (time 1788528127.68 total 20.04 last 14.90)
 - mkdir 464 (time 1788528137.70 total 30.06 last 14.37)
 - mkdir 619 (time 1788528147.73 total 40.09 last 15.46)
 - mkdir 771 (time 1788528157.76 total 50.13 last 15.14)
 - mkdir 937 (time 1788528167.80 total 60.17 last 16.54)
total: 1000 mkdir in 65.43 seconds: 15.28 ops/second
 - mkdir 164 (time 1788528117.62 total 10.02 last 16.37)
 - mkdir 321 (time 1788528127.63 total 20.02 last 15.69)
 - mkdir 465 (time 1788528137.67 total 30.06 last 14.34)
 - mkdir 616 (time 1788528147.69 total 40.09 last 15.07)
 - mkdir 768 (time 1788528157.75 total 50.14 last 15.12)
 - mkdir 933 (time 1788528167.76 total 60.16 last 16.47)
total: 1000 mkdir in 65.52 seconds: 15.26 ops/second
 - mkdir 174 (time 1788528117.75 total 10.04 last 17.33)
 - mkdir 329 (time 1788528127.77 total 20.06 last 15.48)
 - mkdir 467 (time 1788528137.80 total 30.09 last 13.75)
 - mkdir 620 (time 1788528147.83 total 40.11 last 15.26)
 - mkdir 767 (time 1788528157.84 total 50.12 last 14.69)
 - mkdir 935 (time 1788528167.89 total 60.18 last 16.71)
total: 1000 mkdir in 65.42 seconds: 15.29 ops/second
 - mkdir 172 (time 1788528117.64 total 10.07 last 17.08)
 - mkdir 316 (time 1788528127.64 total 20.07 last 14.40)
 - mkdir 456 (time 1788528137.67 total 30.10 last 13.96)
 - mkdir 606 (time 1788528147.68 total 40.11 last 14.99)
 - mkdir 759 (time 1788528157.73 total 50.17 last 15.21)
 - mkdir 929 (time 1788528167.82 total 60.25 last 16.86)
total: 1000 mkdir in 66.56 seconds: 15.02 ops/second
 - mkdir 162 (time 1788528117.57 total 10.02 last 16.17)
 - mkdir 314 (time 1788528127.61 total 20.06 last 15.14)
 - mkdir 452 (time 1788528137.62 total 30.06 last 13.79)
 - mkdir 600 (time 1788528147.64 total 40.08 last 14.77)
 - mkdir 754 (time 1788528157.66 total 50.11 last 15.36)
 - mkdir 922 (time 1788528167.71 total 60.15 last 16.73)
total: 1000 mkdir in 67.12 seconds: 14.90 ops/second
 - mkdir 167 (time 1788528117.87 total 10.03 last 16.65)
 - mkdir 319 (time 1788528127.90 total 20.06 last 15.15)
 - mkdir 453 (time 1788528137.99 total 30.16 last 13.27)
 - mkdir 606 (time 1788528148.04 total 40.20 last 15.23)
 - mkdir 754 (time 1788528158.09 total 50.25 last 14.72)
 - mkdir 922 (time 1788528168.10 total 60.27 last 16.78)
total: 1000 mkdir in 67.04 seconds: 14.92 ops/second
 - mkdir 165 (time 1788528117.38 total 10.06 last 16.41)
 - mkdir 308 (time 1788528127.43 total 20.11 last 14.23)
 - mkdir 439 (time 1788528137.46 total 30.14 last 13.06)
 - mkdir 590 (time 1788528147.51 total 40.19 last 15.02)
 - mkdir 741 (time 1788528157.57 total 50.25 last 15.01)
 - mkdir 911 (time 1788528167.63 total 60.31 last 16.89)
total: 1000 mkdir in 68.09 seconds: 14.69 ops/second
Time for file creation 78 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788528181 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1788528183 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788528183 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788528183 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788528184 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788528184 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788528184 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788528184 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788528184 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1788528184 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
Time for file removal 43 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8