== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 06:08:18 (1788602898)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.56 seconds: 17.71 ops/second
 - mkdir 143 (time 1788602918.36 total 10.07 last 14.20)
 - mkdir 265 (time 1788602928.41 total 20.12 last 12.14)
 - mkdir 412 (time 1788602938.42 total 30.14 last 14.68)
 - mkdir 577 (time 1788602948.53 total 40.24 last 16.33)
 - mkdir 777 (time 1788602958.55 total 50.26 last 19.96)
total: 1000 mkdir in 57.71 seconds: 17.33 ops/second
 - mkdir 147 (time 1788602919.43 total 10.02 last 14.67)
 - mkdir 268 (time 1788602929.44 total 20.03 last 12.09)
 - mkdir 423 (time 1788602939.50 total 30.09 last 15.40)
 - mkdir 585 (time 1788602949.55 total 40.14 last 16.12)
 - mkdir 795 (time 1788602959.60 total 50.19 last 20.91)
total: 1000 mkdir in 57.89 seconds: 17.28 ops/second
 - mkdir 147 (time 1788602919.55 total 10.05 last 14.62)
 - mkdir 268 (time 1788602929.60 total 20.10 last 12.05)
 - mkdir 416 (time 1788602939.64 total 30.14 last 14.73)
 - mkdir 574 (time 1788602949.70 total 40.20 last 15.71)
 - mkdir 794 (time 1788602959.71 total 50.22 last 21.97)
total: 1000 mkdir in 57.81 seconds: 17.30 ops/second
 - mkdir 141 (time 1788602918.87 total 10.05 last 14.03)
 - mkdir 260 (time 1788602928.89 total 20.07 last 11.88)
 - mkdir 408 (time 1788602938.90 total 30.08 last 14.78)
 - mkdir 572 (time 1788602949.01 total 40.20 last 16.21)
 - mkdir 776 (time 1788602959.02 total 50.20 last 20.39)
total: 1000 mkdir in 58.52 seconds: 17.09 ops/second
 - mkdir 143 (time 1788602920.02 total 10.07 last 14.20)
 - mkdir 271 (time 1788602930.02 total 20.08 last 12.79)
 - mkdir 419 (time 1788602940.09 total 30.14 last 14.70)
 - mkdir 580 (time 1788602950.15 total 40.20 last 16.01)
 - mkdir 797 (time 1788602960.15 total 50.21 last 21.68)
total: 1000 mkdir in 57.57 seconds: 17.37 ops/second
 - mkdir 147 (time 1788602919.87 total 10.13 last 14.52)
 - mkdir 271 (time 1788602929.88 total 20.14 last 12.38)
 - mkdir 420 (time 1788602939.89 total 30.15 last 14.89)
 - mkdir 582 (time 1788602949.90 total 40.16 last 16.18)
 - mkdir 791 (time 1788602959.92 total 50.18 last 20.87)
total: 1000 mkdir in 57.98 seconds: 17.25 ops/second
 - mkdir 135 (time 1788602918.56 total 10.01 last 13.49)
 - mkdir 253 (time 1788602928.58 total 20.03 last 11.78)
 - mkdir 395 (time 1788602938.64 total 30.09 last 14.11)
 - mkdir 557 (time 1788602948.73 total 40.17 last 16.07)
 - mkdir 763 (time 1788602958.77 total 50.22 last 20.50)
total: 1000 mkdir in 59.21 seconds: 16.89 ops/second
 - mkdir 143 (time 1788602919.56 total 10.01 last 14.29)
 - mkdir 263 (time 1788602929.62 total 20.06 last 11.93)
 - mkdir 408 (time 1788602939.66 total 30.10 last 14.45)
 - mkdir 572 (time 1788602949.73 total 40.17 last 16.28)
 - mkdir 785 (time 1788602959.74 total 50.18 last 21.28)
total: 1000 mkdir in 58.27 seconds: 17.16 ops/second
 - mkdir 137 (time 1788602918.79 total 10.00 last 13.69)
 - mkdir 253 (time 1788602928.81 total 20.02 last 11.58)
 - mkdir 395 (time 1788602938.87 total 30.08 last 14.12)
 - mkdir 560 (time 1788602948.93 total 40.14 last 16.40)
 - mkdir 757 (time 1788602958.97 total 50.18 last 19.63)
total: 1000 mkdir in 59.12 seconds: 16.91 ops/second
 - mkdir 136 (time 1788602919.14 total 10.10 last 13.46)
 - mkdir 245 (time 1788602929.14 total 20.11 last 10.89)
 - mkdir 385 (time 1788602939.17 total 30.13 last 13.96)
 - mkdir 545 (time 1788602949.20 total 40.17 last 15.95)
 - mkdir 751 (time 1788602959.23 total 50.20 last 20.54)
total: 1000 mkdir in 59.90 seconds: 16.70 ops/second
Time for file creation 70 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 11 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.44 seconds: 22.59 ops/second
 - mkdir 216 (time 1788603461.10 total 10.03 last 21.54)
 - mkdir 409 (time 1788603471.10 total 20.03 last 19.30)
 - mkdir 581 (time 1788603481.10 total 30.03 last 17.20)
 - mkdir 795 (time 1788603491.13 total 40.06 last 21.34)
total: 1000 mkdir in 49.83 seconds: 20.07 ops/second
 - mkdir 217 (time 1788603461.71 total 10.05 last 21.59)
 - mkdir 409 (time 1788603471.76 total 20.10 last 19.10)
 - mkdir 584 (time 1788603481.83 total 30.17 last 17.37)
 - mkdir 811 (time 1788603491.86 total 40.20 last 22.63)
total: 1000 mkdir in 49.38 seconds: 20.25 ops/second
 - mkdir 213 (time 1788603461.41 total 10.04 last 21.22)
 - mkdir 396 (time 1788603471.42 total 20.05 last 18.28)
 - mkdir 577 (time 1788603481.43 total 30.05 last 18.09)
 - mkdir 792 (time 1788603491.44 total 40.07 last 21.47)
 - mkdir 991 (time 1788603501.44 total 50.07 last 19.89)
total: 1000 mkdir in 50.68 seconds: 19.73 ops/second
 - mkdir 212 (time 1788603461.88 total 10.05 last 21.09)
 - mkdir 399 (time 1788603471.89 total 20.06 last 18.68)
 - mkdir 573 (time 1788603481.92 total 30.09 last 17.35)
 - mkdir 794 (time 1788603492.00 total 40.17 last 21.92)
 - mkdir 993 (time 1788603502.08 total 50.25 last 19.75)
total: 1000 mkdir in 50.74 seconds: 19.71 ops/second
 - mkdir 212 (time 1788603461.83 total 10.07 last 21.05)
 - mkdir 405 (time 1788603471.88 total 20.11 last 19.22)
 - mkdir 574 (time 1788603481.88 total 30.12 last 16.89)
 - mkdir 799 (time 1788603491.93 total 40.16 last 22.39)
 - mkdir 992 (time 1788603501.93 total 50.17 last 19.29)
total: 1000 mkdir in 50.87 seconds: 19.66 ops/second
 - mkdir 216 (time 1788603460.94 total 10.05 last 21.48)
 - mkdir 405 (time 1788603471.02 total 20.13 last 18.76)
 - mkdir 573 (time 1788603481.08 total 30.19 last 16.69)
 - mkdir 781 (time 1788603491.10 total 40.21 last 20.76)
 - mkdir 984 (time 1788603501.24 total 50.36 last 20.01)
total: 1000 mkdir in 51.86 seconds: 19.28 ops/second
 - mkdir 216 (time 1788603461.98 total 10.03 last 21.53)
 - mkdir 404 (time 1788603472.03 total 20.08 last 18.70)
 - mkdir 575 (time 1788603482.09 total 30.14 last 17.00)
 - mkdir 795 (time 1788603492.11 total 40.16 last 21.96)
 - mkdir 984 (time 1788603502.12 total 50.17 last 18.88)
total: 1000 mkdir in 51.50 seconds: 19.42 ops/second
 - mkdir 216 (time 1788603461.71 total 10.04 last 21.51)
 - mkdir 404 (time 1788603471.76 total 20.09 last 18.70)
 - mkdir 568 (time 1788603481.78 total 30.11 last 16.37)
 - mkdir 783 (time 1788603491.80 total 40.13 last 21.47)
 - mkdir 976 (time 1788603501.81 total 50.14 last 19.28)
total: 1000 mkdir in 52.02 seconds: 19.22 ops/second
 - mkdir 210 (time 1788603461.36 total 10.03 last 20.94)
 - mkdir 395 (time 1788603471.38 total 20.05 last 18.47)
 - mkdir 564 (time 1788603481.40 total 30.07 last 16.86)
 - mkdir 773 (time 1788603491.43 total 40.10 last 20.84)
 - mkdir 969 (time 1788603501.45 total 50.12 last 19.56)
total: 1000 mkdir in 52.80 seconds: 18.94 ops/second
 - mkdir 197 (time 1788603461.04 total 10.02 last 19.67)
 - mkdir 377 (time 1788603471.09 total 20.06 last 17.92)
 - mkdir 540 (time 1788603481.13 total 30.10 last 16.24)
 - mkdir 743 (time 1788603491.16 total 40.13 last 20.24)
 - mkdir 940 (time 1788603501.44 total 50.41 last 19.16)
total: 1000 mkdir in 54.83 seconds: 18.24 ops/second
Time for file creation 62 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788603511 ; total 0 ; last 0)
total: 1000 unlinks in 21 seconds: 47.619049 unlinks/second
 - unlinked 0 (time 1788603512 ; total 0 ; last 0)
total: 1000 unlinks in 21 seconds: 47.619049 unlinks/second
 - unlinked 0 (time 1788603512 ; total 0 ; last 0)
total: 1000 unlinks in 21 seconds: 47.619049 unlinks/second
 - unlinked 0 (time 1788603512 ; total 0 ; last 0)
total: 1000 unlinks in 21 seconds: 47.619049 unlinks/second
 - unlinked 0 (time 1788603512 ; total 0 ; last 0)
total: 1000 unlinks in 21 seconds: 47.619049 unlinks/second
 - unlinked 0 (time 1788603512 ; total 0 ; last 0)
total: 1000 unlinks in 21 seconds: 47.619049 unlinks/second
 - unlinked 0 (time 1788603512 ; total 0 ; last 0)
total: 1000 unlinks in 21 seconds: 47.619049 unlinks/second
 - unlinked 0 (time 1788603512 ; total 0 ; last 0)
total: 1000 unlinks in 21 seconds: 47.619049 unlinks/second
 - unlinked 0 (time 1788603513 ; total 0 ; last 0)
total: 1000 unlinks in 20 seconds: 50.000000 unlinks/second
 - unlinked 0 (time 1788603513 ; total 0 ; last 0)
total: 1000 unlinks in 21 seconds: 47.619049 unlinks/second
Time for file removal 29 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8