== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 18:53:32 (1788303212)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.45 seconds: 22.24 ops/second
 - mkdir 157 (time 1788303234.59 total 10.01 last 15.69)
 - mkdir 356 (time 1788303244.63 total 20.05 last 19.82)
 - mkdir 546 (time 1788303254.66 total 30.08 last 18.94)
 - mkdir 669 (time 1788303264.68 total 40.10 last 12.27)
 - mkdir 850 (time 1788303274.69 total 50.11 last 18.09)
total: 1000 mkdir in 59.11 seconds: 16.92 ops/second
 - mkdir 179 (time 1788303237.41 total 10.02 last 17.87)
 - mkdir 390 (time 1788303247.42 total 20.02 last 21.09)
 - mkdir 552 (time 1788303257.44 total 30.05 last 16.17)
 - mkdir 684 (time 1788303267.48 total 40.08 last 13.15)
 - mkdir 858 (time 1788303277.51 total 50.11 last 17.34)
 - mkdir 991 (time 1788303287.64 total 60.25 last 13.12)
total: 1000 mkdir in 61.20 seconds: 16.34 ops/second
 - mkdir 171 (time 1788303237.12 total 10.01 last 17.08)
 - mkdir 383 (time 1788303247.17 total 20.06 last 21.10)
 - mkdir 542 (time 1788303257.21 total 30.10 last 15.84)
 - mkdir 677 (time 1788303267.25 total 40.14 last 13.44)
 - mkdir 853 (time 1788303277.27 total 50.16 last 17.57)
 - mkdir 988 (time 1788303287.44 total 60.33 last 13.28)
total: 1000 mkdir in 61.69 seconds: 16.21 ops/second
 - mkdir 189 (time 1788303237.83 total 10.01 last 18.88)
 - mkdir 403 (time 1788303247.86 total 20.04 last 21.34)
 - mkdir 552 (time 1788303257.93 total 30.11 last 14.80)
 - mkdir 694 (time 1788303267.98 total 40.16 last 14.13)
 - mkdir 860 (time 1788303278.04 total 50.22 last 16.50)
 - mkdir 992 (time 1788303288.17 total 60.35 last 13.04)
total: 1000 mkdir in 61.46 seconds: 16.27 ops/second
 - mkdir 164 (time 1788303237.02 total 10.03 last 16.36)
 - mkdir 381 (time 1788303247.07 total 20.08 last 21.58)
 - mkdir 543 (time 1788303257.07 total 30.08 last 16.20)
 - mkdir 675 (time 1788303267.07 total 40.08 last 13.20)
 - mkdir 851 (time 1788303277.14 total 50.15 last 17.48)
 - mkdir 985 (time 1788303287.22 total 60.23 last 13.29)
total: 1000 mkdir in 62.33 seconds: 16.04 ops/second
 - mkdir 176 (time 1788303237.76 total 10.01 last 17.58)
 - mkdir 393 (time 1788303247.78 total 20.03 last 21.66)
 - mkdir 545 (time 1788303257.78 total 30.04 last 15.19)
 - mkdir 681 (time 1788303267.80 total 40.06 last 13.57)
 - mkdir 850 (time 1788303277.85 total 50.10 last 16.82)
 - mkdir 980 (time 1788303287.97 total 60.22 last 12.85)
total: 1000 mkdir in 62.97 seconds: 15.88 ops/second
 - mkdir 181 (time 1788303237.80 total 10.01 last 18.09)
 - mkdir 394 (time 1788303247.81 total 20.01 last 21.30)
 - mkdir 543 (time 1788303257.84 total 30.04 last 14.85)
 - mkdir 680 (time 1788303267.89 total 40.09 last 13.63)
 - mkdir 850 (time 1788303277.98 total 50.18 last 16.85)
 - mkdir 980 (time 1788303288.05 total 60.25 last 12.91)
total: 1000 mkdir in 63.15 seconds: 15.83 ops/second
 - mkdir 159 (time 1788303236.32 total 10.05 last 15.82)
 - mkdir 366 (time 1788303246.36 total 20.09 last 20.61)
 - mkdir 525 (time 1788303256.47 total 30.20 last 15.72)
 - mkdir 653 (time 1788303266.48 total 40.21 last 12.79)
 - mkdir 822 (time 1788303276.50 total 50.23 last 16.87)
 - mkdir 966 (time 1788303286.51 total 60.25 last 14.38)
total: 1000 mkdir in 65.57 seconds: 15.25 ops/second
 - mkdir 174 (time 1788303237.59 total 10.00 last 17.39)
 - mkdir 382 (time 1788303247.63 total 20.04 last 20.73)
 - mkdir 534 (time 1788303257.68 total 30.09 last 15.12)
 - mkdir 670 (time 1788303267.72 total 40.13 last 13.55)
 - mkdir 843 (time 1788303277.78 total 50.19 last 17.19)
 - mkdir 971 (time 1788303287.80 total 60.21 last 12.78)
total: 1000 mkdir in 64.89 seconds: 15.41 ops/second
 - mkdir 166 (time 1788303237.63 total 10.01 last 16.58)
 - mkdir 368 (time 1788303247.63 total 20.02 last 20.19)
 - mkdir 506 (time 1788303257.63 total 30.02 last 13.80)
 - mkdir 640 (time 1788303267.67 total 40.06 last 13.35)
 - mkdir 812 (time 1788303277.73 total 50.12 last 17.10)
 - mkdir 945 (time 1788303287.83 total 60.21 last 13.17)
total: 1000 mkdir in 67.75 seconds: 14.76 ops/second
Time for file creation 80 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 17 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.59 seconds: 17.01 ops/second
 - mkdir 173 (time 1788303814.76 total 10.07 last 17.18)
 - mkdir 354 (time 1788303824.78 total 20.10 last 18.05)
 - mkdir 546 (time 1788303834.79 total 30.10 last 19.19)
 - mkdir 731 (time 1788303844.81 total 40.12 last 18.46)
 - mkdir 968 (time 1788303854.82 total 50.14 last 23.67)
total: 1000 mkdir in 51.54 seconds: 19.40 ops/second
 - mkdir 151 (time 1788303813.00 total 10.01 last 15.09)
 - mkdir 329 (time 1788303823.00 total 20.01 last 17.79)
 - mkdir 506 (time 1788303833.01 total 30.02 last 17.69)
 - mkdir 689 (time 1788303843.05 total 40.06 last 18.23)
 - mkdir 927 (time 1788303853.05 total 50.06 last 23.80)
total: 1000 mkdir in 53.79 seconds: 18.59 ops/second
 - mkdir 162 (time 1788303813.94 total 10.02 last 16.17)
 - mkdir 339 (time 1788303823.97 total 20.05 last 17.64)
 - mkdir 524 (time 1788303834.01 total 30.09 last 18.43)
 - mkdir 711 (time 1788303844.02 total 40.10 last 18.68)
 - mkdir 937 (time 1788303854.05 total 50.13 last 22.53)
total: 1000 mkdir in 53.69 seconds: 18.63 ops/second
 - mkdir 168 (time 1788303814.33 total 10.07 last 16.69)
 - mkdir 348 (time 1788303824.38 total 20.12 last 17.91)
 - mkdir 536 (time 1788303834.38 total 30.12 last 18.79)
 - mkdir 716 (time 1788303844.42 total 40.16 last 17.93)
 - mkdir 946 (time 1788303854.43 total 50.17 last 22.98)
total: 1000 mkdir in 53.49 seconds: 18.69 ops/second
 - mkdir 160 (time 1788303813.80 total 10.05 last 15.92)
 - mkdir 339 (time 1788303823.84 total 20.10 last 17.82)
 - mkdir 521 (time 1788303833.92 total 30.17 last 18.06)
 - mkdir 700 (time 1788303843.93 total 40.18 last 17.88)
 - mkdir 935 (time 1788303854.00 total 50.25 last 23.34)
total: 1000 mkdir in 54.08 seconds: 18.49 ops/second
 - mkdir 147 (time 1788303812.42 total 10.07 last 14.60)
 - mkdir 315 (time 1788303822.46 total 20.11 last 16.73)
 - mkdir 486 (time 1788303832.47 total 30.12 last 17.08)
 - mkdir 675 (time 1788303842.56 total 40.21 last 18.73)
 - mkdir 908 (time 1788303852.64 total 50.28 last 23.12)
total: 1000 mkdir in 56.26 seconds: 17.77 ops/second
 - mkdir 150 (time 1788303813.60 total 10.01 last 14.99)
 - mkdir 332 (time 1788303823.64 total 20.05 last 18.12)
 - mkdir 519 (time 1788303833.73 total 30.14 last 18.54)
 - mkdir 697 (time 1788303843.73 total 40.14 last 17.79)
 - mkdir 924 (time 1788303853.78 total 50.19 last 22.59)
total: 1000 mkdir in 55.12 seconds: 18.14 ops/second
 - mkdir 146 (time 1788303811.83 total 10.00 last 14.60)
 - mkdir 318 (time 1788303821.84 total 20.01 last 17.18)
 - mkdir 485 (time 1788303831.87 total 30.05 last 16.64)
 - mkdir 677 (time 1788303841.93 total 40.11 last 19.09)
 - mkdir 893 (time 1788303851.95 total 50.13 last 21.56)
total: 1000 mkdir in 57.41 seconds: 17.42 ops/second
 - mkdir 154 (time 1788303813.94 total 10.04 last 15.34)
 - mkdir 330 (time 1788303823.98 total 20.08 last 17.53)
 - mkdir 517 (time 1788303833.98 total 30.08 last 18.69)
 - mkdir 692 (time 1788303843.99 total 40.10 last 17.48)
 - mkdir 923 (time 1788303854.02 total 50.13 last 23.03)
total: 1000 mkdir in 55.41 seconds: 18.05 ops/second
 - mkdir 165 (time 1788303814.50 total 10.01 last 16.48)
 - mkdir 337 (time 1788303824.52 total 20.03 last 17.17)
 - mkdir 524 (time 1788303834.52 total 30.04 last 18.68)
 - mkdir 706 (time 1788303844.54 total 40.05 last 18.18)
 - mkdir 933 (time 1788303854.54 total 50.05 last 22.68)
total: 1000 mkdir in 54.93 seconds: 18.20 ops/second
Time for file creation 67 sec for 1 RPC sent at a time
 - unlinked 0 (time 1788303867 ; total 0 ; last 0)
total: 1000 unlinks in 21 seconds: 47.619049 unlinks/second
 - unlinked 0 (time 1788303867 ; total 0 ; last 0)
total: 1000 unlinks in 21 seconds: 47.619049 unlinks/second
 - unlinked 0 (time 1788303868 ; total 0 ; last 0)
total: 1000 unlinks in 21 seconds: 47.619049 unlinks/second
 - unlinked 0 (time 1788303868 ; total 0 ; last 0)
total: 1000 unlinks in 21 seconds: 47.619049 unlinks/second
 - unlinked 0 (time 1788303868 ; total 0 ; last 0)
total: 1000 unlinks in 22 seconds: 45.454544 unlinks/second
 - unlinked 0 (time 1788303868 ; total 0 ; last 0)
total: 1000 unlinks in 22 seconds: 45.454544 unlinks/second
 - unlinked 0 (time 1788303868 ; total 0 ; last 0)
total: 1000 unlinks in 22 seconds: 45.454544 unlinks/second
 - unlinked 0 (time 1788303869 ; total 1 ; last 1)
total: 1000 unlinks in 22 seconds: 45.454544 unlinks/second
 - unlinked 0 (time 1788303869 ; total 0 ; last 0)
total: 1000 unlinks in 21 seconds: 47.619049 unlinks/second
 - unlinked 0 (time 1788303869 ; total 0 ; last 0)
total: 1000 unlinks in 21 seconds: 47.619049 unlinks/second
Time for file removal 33 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8