== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 19:26:57 (1788218817)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.52 seconds: 19.24 ops/second
 - mkdir 165 (time 1788218836.47 total 10.03 last 16.44)
 - mkdir 318 (time 1788218846.49 total 20.05 last 15.27)
 - mkdir 469 (time 1788218856.50 total 30.06 last 15.08)
 - mkdir 634 (time 1788218866.52 total 40.08 last 16.47)
 - mkdir 802 (time 1788218876.56 total 50.12 last 16.73)
 - mkdir 973 (time 1788218886.62 total 60.18 last 17.00)
total: 1000 mkdir in 61.62 seconds: 16.23 ops/second
 - mkdir 159 (time 1788218837.43 total 10.04 last 15.84)
 - mkdir 306 (time 1788218847.45 total 20.07 last 14.66)
 - mkdir 453 (time 1788218857.51 total 30.12 last 14.62)
 - mkdir 618 (time 1788218867.60 total 40.21 last 16.34)
 - mkdir 785 (time 1788218877.64 total 50.25 last 16.64)
 - mkdir 947 (time 1788218887.66 total 60.27 last 16.17)
total: 1000 mkdir in 66.09 seconds: 15.13 ops/second
 - mkdir 159 (time 1788218837.09 total 10.01 last 15.88)
 - mkdir 309 (time 1788218847.13 total 20.06 last 14.93)
 - mkdir 452 (time 1788218857.19 total 30.12 last 14.21)
 - mkdir 620 (time 1788218867.22 total 40.15 last 16.75)
 - mkdir 782 (time 1788218877.24 total 50.16 last 16.17)
 - mkdir 944 (time 1788218887.29 total 60.21 last 16.13)
total: 1000 mkdir in 66.46 seconds: 15.05 ops/second
 - mkdir 161 (time 1788218837.37 total 10.02 last 16.07)
 - mkdir 307 (time 1788218847.49 total 20.14 last 14.43)
 - mkdir 457 (time 1788218857.52 total 30.17 last 14.96)
 - mkdir 619 (time 1788218867.55 total 40.20 last 16.15)
 - mkdir 782 (time 1788218877.57 total 50.22 last 16.26)
 - mkdir 947 (time 1788218887.61 total 60.26 last 16.43)
total: 1000 mkdir in 66.65 seconds: 15.00 ops/second
 - mkdir 157 (time 1788218837.45 total 10.01 last 15.68)
 - mkdir 305 (time 1788218847.48 total 20.04 last 14.76)
 - mkdir 450 (time 1788218857.52 total 30.08 last 14.44)
 - mkdir 614 (time 1788218867.55 total 40.12 last 16.34)
 - mkdir 775 (time 1788218877.56 total 50.12 last 16.09)
 - mkdir 942 (time 1788218887.62 total 60.19 last 16.59)
total: 1000 mkdir in 67.26 seconds: 14.87 ops/second
 - mkdir 160 (time 1788218837.25 total 10.08 last 15.87)
 - mkdir 306 (time 1788218847.33 total 20.16 last 14.49)
 - mkdir 453 (time 1788218857.40 total 30.23 last 14.60)
 - mkdir 617 (time 1788218867.47 total 40.30 last 16.27)
 - mkdir 779 (time 1788218877.49 total 50.32 last 16.17)
 - mkdir 939 (time 1788218887.54 total 60.37 last 15.92)
total: 1000 mkdir in 67.65 seconds: 14.78 ops/second
 - mkdir 166 (time 1788218836.52 total 10.02 last 16.57)
 - mkdir 308 (time 1788218846.57 total 20.07 last 14.13)
 - mkdir 454 (time 1788218856.69 total 30.19 last 14.43)
 - mkdir 614 (time 1788218866.71 total 40.20 last 15.97)
 - mkdir 764 (time 1788218876.73 total 50.23 last 14.97)
 - mkdir 923 (time 1788218886.73 total 60.23 last 15.89)
total: 1000 mkdir in 69.13 seconds: 14.47 ops/second
 - mkdir 153 (time 1788218836.25 total 10.03 last 15.25)
 - mkdir 302 (time 1788218846.29 total 20.07 last 14.84)
 - mkdir 441 (time 1788218856.35 total 30.13 last 13.82)
 - mkdir 598 (time 1788218866.38 total 40.16 last 15.65)
 - mkdir 749 (time 1788218876.40 total 50.18 last 15.06)
 - mkdir 907 (time 1788218886.43 total 60.21 last 15.76)
total: 1000 mkdir in 69.62 seconds: 14.36 ops/second
 - mkdir 161 (time 1788218837.35 total 10.02 last 16.07)
 - mkdir 310 (time 1788218847.39 total 20.06 last 14.84)
 - mkdir 453 (time 1788218857.40 total 30.07 last 14.28)
 - mkdir 614 (time 1788218867.47 total 40.14 last 15.99)
 - mkdir 772 (time 1788218877.49 total 50.16 last 15.77)
 - mkdir 929 (time 1788218887.55 total 60.22 last 15.60)
total: 1000 mkdir in 68.79 seconds: 14.54 ops/second
 - mkdir 152 (time 1788218837.26 total 10.04 last 15.14)
 - mkdir 289 (time 1788218847.29 total 20.07 last 13.66)
 - mkdir 422 (time 1788218857.30 total 30.08 last 13.29)
 - mkdir 580 (time 1788218867.36 total 40.15 last 15.69)
 - mkdir 736 (time 1788218877.41 total 50.19 last 15.53)
 - mkdir 893 (time 1788218887.48 total 60.26 last 15.59)
 - mkdir 982 (time 1788218897.49 total 70.27 last 8.89)
total: 1000 mkdir in 71.09 seconds: 14.07 ops/second
Time for file creation 79 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 1 seconds: 0.000000 unlinks/second
Time for file removal 10 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.58 seconds: 17.19 ops/second
 - mkdir 166 (time 1788219400.42 total 10.01 last 16.59)
 - mkdir 359 (time 1788219410.44 total 20.03 last 19.26)
 - mkdir 535 (time 1788219420.50 total 30.08 last 17.51)
 - mkdir 715 (time 1788219430.51 total 40.10 last 17.98)
 - mkdir 868 (time 1788219440.52 total 50.11 last 15.28)
total: 1000 mkdir in 56.71 seconds: 17.63 ops/second
 - mkdir 171 (time 1788219401.56 total 10.14 last 16.86)
 - mkdir 369 (time 1788219411.59 total 20.17 last 19.75)
 - mkdir 532 (time 1788219421.63 total 30.22 last 16.23)
 - mkdir 704 (time 1788219431.74 total 40.32 last 17.02)
 - mkdir 859 (time 1788219441.78 total 50.36 last 15.45)
total: 1000 mkdir in 59.69 seconds: 16.75 ops/second
 - mkdir 174 (time 1788219402.31 total 10.02 last 17.36)
 - mkdir 365 (time 1788219412.35 total 20.06 last 19.02)
 - mkdir 528 (time 1788219422.36 total 30.07 last 16.29)
 - mkdir 700 (time 1788219432.47 total 40.18 last 17.01)
 - mkdir 862 (time 1788219442.52 total 50.23 last 16.12)
total: 1000 mkdir in 59.61 seconds: 16.78 ops/second
 - mkdir 182 (time 1788219402.70 total 10.02 last 18.16)
 - mkdir 378 (time 1788219412.73 total 20.05 last 19.55)
 - mkdir 535 (time 1788219422.75 total 30.07 last 15.67)
 - mkdir 694 (time 1788219432.77 total 40.09 last 15.87)
 - mkdir 855 (time 1788219442.81 total 50.13 last 16.03)
total: 1000 mkdir in 59.88 seconds: 16.70 ops/second
 - mkdir 130 (time 1788219398.13 total 10.02 last 12.97)
 - mkdir 325 (time 1788219408.16 total 20.05 last 19.44)
 - mkdir 490 (time 1788219418.20 total 30.09 last 16.44)
 - mkdir 655 (time 1788219428.23 total 40.11 last 16.46)
 - mkdir 783 (time 1788219438.26 total 50.15 last 12.76)
 - mkdir 960 (time 1788219448.32 total 60.21 last 17.59)
total: 1000 mkdir in 64.85 seconds: 15.42 ops/second
 - mkdir 173 (time 1788219402.21 total 10.02 last 17.27)
 - mkdir 365 (time 1788219412.21 total 20.02 last 19.19)
 - mkdir 522 (time 1788219422.24 total 30.05 last 15.66)
 - mkdir 688 (time 1788219432.30 total 40.11 last 16.49)
 - mkdir 842 (time 1788219442.34 total 50.15 last 15.34)
 - mkdir 990 (time 1788219452.38 total 60.19 last 14.75)
total: 1000 mkdir in 61.21 seconds: 16.34 ops/second
 - mkdir 167 (time 1788219402.00 total 10.01 last 16.68)
 - mkdir 362 (time 1788219412.05 total 20.06 last 19.41)
 - mkdir 525 (time 1788219422.17 total 30.18 last 16.11)
 - mkdir 688 (time 1788219432.26 total 40.28 last 16.15)
 - mkdir 845 (time 1788219442.28 total 50.30 last 15.67)
 - mkdir 989 (time 1788219452.36 total 60.38 last 14.29)
total: 1000 mkdir in 61.88 seconds: 16.16 ops/second
 - mkdir 179 (time 1788219402.50 total 10.02 last 17.87)
 - mkdir 371 (time 1788219412.52 total 20.04 last 19.16)
 - mkdir 528 (time 1788219422.56 total 30.07 last 15.64)
 - mkdir 692 (time 1788219432.57 total 40.08 last 16.38)
 - mkdir 846 (time 1788219442.63 total 50.14 last 15.31)
 - mkdir 988 (time 1788219452.66 total 60.17 last 14.16)
total: 1000 mkdir in 61.52 seconds: 16.26 ops/second
 - mkdir 164 (time 1788219401.34 total 10.01 last 16.38)
 - mkdir 346 (time 1788219411.35 total 20.03 last 18.18)
 - mkdir 504 (time 1788219421.45 total 30.12 last 15.65)
 - mkdir 678 (time 1788219431.48 total 40.15 last 17.35)
 - mkdir 830 (time 1788219441.57 total 50.24 last 15.07)
 - mkdir 978 (time 1788219451.66 total 60.33 last 14.67)
total: 1000 mkdir in 62.91 seconds: 15.89 ops/second
 - mkdir 172 (time 1788219402.52 total 10.01 last 17.18)
 - mkdir 358 (time 1788219412.55 total 20.04 last 18.55)
 - mkdir 510 (time 1788219422.57 total 30.06 last 15.17)
 - mkdir 667 (time 1788219432.61 total 40.10 last 15.64)
 - mkdir 825 (time 1788219442.62 total 50.11 last 15.79)
 - mkdir 965 (time 1788219452.69 total 60.18 last 13.90)
total: 1000 mkdir in 63.26 seconds: 15.81 ops/second
Time for file creation 76 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788219463 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788219464 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788219464 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788219464 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788219464 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788219465 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1788219464 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788219464 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788219465 ; total 1 ; last 1)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1788219465 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
Time for file removal 43 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8