== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 09:11:54 (1787145114)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.51 seconds: 19.43 ops/second
 - mkdir 110 (time 1787145137.56 total 10.04 last 10.96)
 - mkdir 264 (time 1787145147.59 total 20.08 last 15.34)
 - mkdir 423 (time 1787145157.63 total 30.11 last 15.85)
 - mkdir 573 (time 1787145167.67 total 40.15 last 14.94)
 - mkdir 727 (time 1787145177.70 total 50.18 last 15.36)
 - mkdir 902 (time 1787145187.78 total 60.26 last 17.36)
total: 1000 mkdir in 65.19 seconds: 15.34 ops/second
 - mkdir 116 (time 1787145138.01 total 10.00 last 11.60)
 - mkdir 273 (time 1787145148.06 total 20.06 last 15.61)
 - mkdir 430 (time 1787145158.09 total 30.08 last 15.67)
 - mkdir 580 (time 1787145168.11 total 40.11 last 14.96)
 - mkdir 739 (time 1787145178.12 total 50.12 last 15.89)
 - mkdir 906 (time 1787145188.13 total 60.13 last 16.68)
total: 1000 mkdir in 65.01 seconds: 15.38 ops/second
 - mkdir 116 (time 1787145139.55 total 10.04 last 11.56)
 - mkdir 266 (time 1787145149.61 total 20.09 last 14.92)
 - mkdir 421 (time 1787145159.63 total 30.11 last 15.47)
 - mkdir 582 (time 1787145169.64 total 40.13 last 16.07)
 - mkdir 737 (time 1787145179.69 total 50.17 last 15.43)
 - mkdir 915 (time 1787145189.72 total 60.20 last 17.75)
total: 1000 mkdir in 66.18 seconds: 15.11 ops/second
 - mkdir 125 (time 1787145140.62 total 10.04 last 12.46)
 - mkdir 278 (time 1787145150.65 total 20.07 last 15.25)
 - mkdir 435 (time 1787145160.66 total 30.08 last 15.68)
 - mkdir 595 (time 1787145170.75 total 40.17 last 15.86)
 - mkdir 749 (time 1787145180.78 total 50.19 last 15.36)
 - mkdir 927 (time 1787145190.83 total 60.25 last 17.70)
total: 1000 mkdir in 67.36 seconds: 14.85 ops/second
 - mkdir 120 (time 1787145140.14 total 10.07 last 11.91)
 - mkdir 274 (time 1787145150.14 total 20.08 last 15.40)
 - mkdir 427 (time 1787145160.16 total 30.09 last 15.28)
 - mkdir 586 (time 1787145170.17 total 40.10 last 15.88)
 - mkdir 737 (time 1787145180.18 total 50.11 last 15.09)
 - mkdir 909 (time 1787145190.21 total 60.15 last 17.14)
total: 1000 mkdir in 68.49 seconds: 14.60 ops/second
 - mkdir 125 (time 1787145140.64 total 10.01 last 12.49)
 - mkdir 275 (time 1787145150.68 total 20.05 last 14.94)
 - mkdir 434 (time 1787145160.69 total 30.06 last 15.88)
 - mkdir 593 (time 1787145170.78 total 40.15 last 15.76)
 - mkdir 749 (time 1787145180.82 total 50.19 last 15.53)
 - mkdir 919 (time 1787145190.94 total 60.30 last 16.81)
total: 1000 mkdir in 68.17 seconds: 14.67 ops/second
 - mkdir 121 (time 1787145139.84 total 10.09 last 11.99)
 - mkdir 270 (time 1787145149.92 total 20.17 last 14.78)
 - mkdir 422 (time 1787145159.96 total 30.21 last 15.14)
 - mkdir 584 (time 1787145170.02 total 40.27 last 16.11)
 - mkdir 738 (time 1787145180.05 total 50.31 last 15.34)
 - mkdir 908 (time 1787145190.12 total 60.37 last 16.89)
total: 1000 mkdir in 69.49 seconds: 14.39 ops/second
 - mkdir 125 (time 1787145140.45 total 10.03 last 12.46)
 - mkdir 278 (time 1787145150.50 total 20.08 last 15.22)
 - mkdir 431 (time 1787145160.55 total 30.13 last 15.23)
 - mkdir 582 (time 1787145170.63 total 40.21 last 14.98)
 - mkdir 734 (time 1787145180.67 total 50.25 last 15.13)
 - mkdir 904 (time 1787145190.70 total 60.28 last 16.96)
total: 1000 mkdir in 69.54 seconds: 14.38 ops/second
 - mkdir 112 (time 1787145140.05 total 10.04 last 11.15)
 - mkdir 262 (time 1787145150.10 total 20.09 last 14.93)
 - mkdir 413 (time 1787145160.11 total 30.10 last 15.09)
 - mkdir 569 (time 1787145170.16 total 40.15 last 15.52)
 - mkdir 723 (time 1787145180.17 total 50.16 last 15.40)
 - mkdir 898 (time 1787145190.20 total 60.19 last 17.44)
total: 1000 mkdir in 70.07 seconds: 14.27 ops/second
 - mkdir 112 (time 1787145140.45 total 10.03 last 11.17)
 - mkdir 253 (time 1787145150.47 total 20.05 last 14.07)
 - mkdir 402 (time 1787145160.49 total 30.06 last 14.88)
 - mkdir 560 (time 1787145170.53 total 40.10 last 15.74)
 - mkdir 711 (time 1787145180.57 total 50.15 last 15.04)
 - mkdir 880 (time 1787145190.59 total 60.17 last 16.86)
 - mkdir 976 (time 1787145200.60 total 70.18 last 9.59)
total: 1000 mkdir in 72.08 seconds: 13.87 ops/second
Time for file creation 85 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 17 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.66 seconds: 15.10 ops/second
 - mkdir 121 (time 1787145719.11 total 10.03 last 12.06)
 - mkdir 309 (time 1787145729.13 total 20.05 last 18.76)
 - mkdir 447 (time 1787145739.14 total 30.06 last 13.79)
 - mkdir 578 (time 1787145749.21 total 40.13 last 13.01)
 - mkdir 699 (time 1787145759.24 total 50.16 last 12.06)
 - mkdir 846 (time 1787145769.31 total 60.22 last 14.61)
 - mkdir 995 (time 1787145779.35 total 70.27 last 14.84)
total: 1000 mkdir in 70.43 seconds: 14.20 ops/second
 - mkdir 147 (time 1787145722.38 total 10.01 last 14.69)
 - mkdir 324 (time 1787145732.38 total 20.01 last 17.69)
 - mkdir 468 (time 1787145742.44 total 30.07 last 14.32)
 - mkdir 575 (time 1787145752.47 total 40.10 last 10.66)
 - mkdir 716 (time 1787145762.49 total 50.12 last 14.07)
 - mkdir 859 (time 1787145772.52 total 60.15 last 14.26)
 - mkdir 986 (time 1787145782.53 total 70.16 last 12.69)
total: 1000 mkdir in 71.68 seconds: 13.95 ops/second
 - mkdir 135 (time 1787145721.25 total 10.03 last 13.46)
 - mkdir 310 (time 1787145731.29 total 20.07 last 17.43)
 - mkdir 450 (time 1787145741.31 total 30.09 last 13.97)
 - mkdir 572 (time 1787145751.34 total 40.12 last 12.16)
 - mkdir 703 (time 1787145761.34 total 50.12 last 13.10)
 - mkdir 850 (time 1787145771.36 total 60.14 last 14.67)
 - mkdir 977 (time 1787145781.41 total 70.19 last 12.63)
total: 1000 mkdir in 72.89 seconds: 13.72 ops/second
 - mkdir 144 (time 1787145722.30 total 10.02 last 14.37)
 - mkdir 321 (time 1787145732.31 total 20.04 last 17.68)
 - mkdir 457 (time 1787145742.35 total 30.08 last 13.54)
 - mkdir 565 (time 1787145752.42 total 40.15 last 10.73)
 - mkdir 708 (time 1787145762.42 total 50.15 last 14.29)
 - mkdir 853 (time 1787145772.45 total 60.18 last 14.46)
 - mkdir 981 (time 1787145782.49 total 70.21 last 12.76)
total: 1000 mkdir in 72.74 seconds: 13.75 ops/second
 - mkdir 157 (time 1787145723.44 total 10.04 last 15.63)
 - mkdir 327 (time 1787145733.57 total 20.17 last 16.79)
 - mkdir 465 (time 1787145743.67 total 30.27 last 13.66)
 - mkdir 578 (time 1787145753.69 total 40.29 last 11.28)
 - mkdir 724 (time 1787145763.74 total 50.34 last 14.53)
 - mkdir 867 (time 1787145773.79 total 60.38 last 14.23)
 - mkdir 989 (time 1787145783.84 total 70.44 last 12.13)
total: 1000 mkdir in 72.07 seconds: 13.88 ops/second
 - mkdir 161 (time 1787145723.87 total 10.04 last 16.03)
 - mkdir 327 (time 1787145733.88 total 20.05 last 16.59)
 - mkdir 463 (time 1787145743.94 total 30.11 last 13.52)
 - mkdir 574 (time 1787145754.04 total 40.21 last 10.99)
 - mkdir 722 (time 1787145764.08 total 50.25 last 14.74)
 - mkdir 863 (time 1787145774.18 total 60.34 last 13.97)
 - mkdir 973 (time 1787145784.32 total 70.49 last 10.85)
total: 1000 mkdir in 74.20 seconds: 13.48 ops/second
 - mkdir 153 (time 1787145723.87 total 10.02 last 15.26)
 - mkdir 320 (time 1787145733.96 total 20.12 last 16.55)
 - mkdir 456 (time 1787145743.97 total 30.13 last 13.59)
 - mkdir 568 (time 1787145754.06 total 40.22 last 11.10)
 - mkdir 713 (time 1787145764.10 total 50.26 last 14.45)
 - mkdir 851 (time 1787145774.11 total 60.26 last 13.79)
 - mkdir 961 (time 1787145784.15 total 70.30 last 10.96)
total: 1000 mkdir in 75.01 seconds: 13.33 ops/second
 - mkdir 147 (time 1787145723.63 total 10.06 last 14.62)
 - mkdir 315 (time 1787145733.69 total 20.13 last 16.69)
 - mkdir 449 (time 1787145743.89 total 30.32 last 13.15)
 - mkdir 563 (time 1787145753.99 total 40.42 last 11.29)
 - mkdir 704 (time 1787145764.00 total 50.44 last 14.08)
 - mkdir 845 (time 1787145774.01 total 60.44 last 14.10)
 - mkdir 954 (time 1787145784.05 total 70.48 last 10.86)
total: 1000 mkdir in 75.98 seconds: 13.16 ops/second
 - mkdir 134 (time 1787145722.86 total 10.00 last 13.40)
 - mkdir 295 (time 1787145732.87 total 20.01 last 16.08)
 - mkdir 430 (time 1787145742.97 total 30.12 last 13.36)
 - mkdir 539 (time 1787145753.02 total 40.16 last 10.85)
 - mkdir 678 (time 1787145763.04 total 50.18 last 13.88)
 - mkdir 822 (time 1787145773.07 total 60.22 last 14.35)
 - mkdir 938 (time 1787145783.17 total 70.31 last 11.49)
total: 1000 mkdir in 77.52 seconds: 12.90 ops/second
 - mkdir 133 (time 1787145722.70 total 10.04 last 13.25)
 - mkdir 299 (time 1787145732.71 total 20.04 last 16.59)
 - mkdir 428 (time 1787145742.75 total 30.08 last 12.85)
 - mkdir 540 (time 1787145752.78 total 40.11 last 11.17)
 - mkdir 675 (time 1787145762.83 total 50.17 last 13.43)
 - mkdir 816 (time 1787145772.86 total 60.19 last 14.06)
 - mkdir 935 (time 1787145782.98 total 70.31 last 11.76)
total: 1000 mkdir in 77.74 seconds: 12.86 ops/second
Time for file creation 90 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787145797 ; total 0 ; last 0)
total: 1000 unlinks in 28 seconds: 35.714287 unlinks/second
 - unlinked 0 (time 1787145799 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1787145800 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1787145800 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787145800 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787145801 ; total 0 ; last 0)
total: 1000 unlinks in 30 seconds: 33.333332 unlinks/second
 - unlinked 0 (time 1787145800 ; total 0 ; last 0)
total: 1000 unlinks in 31 seconds: 32.258064 unlinks/second
 - unlinked 0 (time 1787145800 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1787145800 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1787145801 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
Time for file removal 43 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8