== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 15:39:15 (1782243555)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.41 seconds: 24.47 ops/second
 - mkdir 191 (time 1782243575.27 total 10.03 last 19.04)
 - mkdir 350 (time 1782243585.29 total 20.05 last 15.87)
 - mkdir 511 (time 1782243595.33 total 30.09 last 16.04)
 - mkdir 680 (time 1782243605.34 total 40.10 last 16.88)
 - mkdir 861 (time 1782243615.37 total 50.13 last 18.05)
total: 1000 mkdir in 57.71 seconds: 17.33 ops/second
 - mkdir 185 (time 1782243575.58 total 10.03 last 18.45)
 - mkdir 340 (time 1782243585.65 total 20.10 last 15.39)
 - mkdir 503 (time 1782243595.71 total 30.15 last 16.21)
 - mkdir 666 (time 1782243605.78 total 40.23 last 16.18)
 - mkdir 846 (time 1782243615.81 total 50.25 last 17.95)
total: 1000 mkdir in 58.97 seconds: 16.96 ops/second
 - mkdir 187 (time 1782243575.88 total 10.06 last 18.58)
 - mkdir 346 (time 1782243585.90 total 20.09 last 15.86)
 - mkdir 508 (time 1782243595.95 total 30.14 last 16.12)
 - mkdir 673 (time 1782243605.96 total 40.15 last 16.48)
 - mkdir 855 (time 1782243616.03 total 50.22 last 18.08)
total: 1000 mkdir in 59.33 seconds: 16.85 ops/second
 - mkdir 184 (time 1782243576.20 total 10.05 last 18.31)
 - mkdir 347 (time 1782243586.22 total 20.07 last 16.26)
 - mkdir 502 (time 1782243596.22 total 30.08 last 15.50)
 - mkdir 666 (time 1782243606.31 total 40.16 last 16.26)
 - mkdir 852 (time 1782243616.34 total 50.19 last 18.54)
total: 1000 mkdir in 59.07 seconds: 16.93 ops/second
 - mkdir 188 (time 1782243575.96 total 10.00 last 18.80)
 - mkdir 343 (time 1782243585.99 total 20.02 last 15.46)
 - mkdir 506 (time 1782243596.00 total 30.04 last 16.28)
 - mkdir 675 (time 1782243606.03 total 40.07 last 16.84)
 - mkdir 850 (time 1782243616.04 total 50.08 last 17.49)
total: 1000 mkdir in 59.58 seconds: 16.79 ops/second
 - mkdir 182 (time 1782243575.97 total 10.01 last 18.18)
 - mkdir 340 (time 1782243585.98 total 20.03 last 15.77)
 - mkdir 501 (time 1782243596.03 total 30.07 last 16.03)
 - mkdir 667 (time 1782243606.13 total 40.17 last 16.43)
 - mkdir 842 (time 1782243616.16 total 50.20 last 17.45)
 - mkdir 998 (time 1782243626.23 total 60.28 last 15.48)
total: 1000 mkdir in 60.44 seconds: 16.55 ops/second
 - mkdir 180 (time 1782243575.26 total 10.04 last 17.92)
 - mkdir 337 (time 1782243585.31 total 20.10 last 15.62)
 - mkdir 495 (time 1782243595.33 total 30.11 last 15.77)
 - mkdir 656 (time 1782243605.37 total 40.16 last 16.03)
 - mkdir 832 (time 1782243615.39 total 50.17 last 17.57)
 - mkdir 992 (time 1782243625.44 total 60.22 last 15.92)
total: 1000 mkdir in 61.24 seconds: 16.33 ops/second
 - mkdir 184 (time 1782243576.20 total 10.03 last 18.35)
 - mkdir 345 (time 1782243586.22 total 20.05 last 16.06)
 - mkdir 496 (time 1782243596.32 total 30.15 last 14.95)
 - mkdir 658 (time 1782243606.35 total 40.18 last 16.16)
 - mkdir 836 (time 1782243616.37 total 50.20 last 17.76)
 - mkdir 992 (time 1782243626.45 total 60.28 last 15.48)
total: 1000 mkdir in 60.96 seconds: 16.40 ops/second
 - mkdir 185 (time 1782243575.52 total 10.04 last 18.42)
 - mkdir 339 (time 1782243585.57 total 20.09 last 15.32)
 - mkdir 493 (time 1782243595.61 total 30.13 last 15.34)
 - mkdir 655 (time 1782243605.65 total 40.17 last 16.14)
 - mkdir 831 (time 1782243615.67 total 50.20 last 17.56)
 - mkdir 983 (time 1782243625.78 total 60.30 last 15.05)
total: 1000 mkdir in 61.84 seconds: 16.17 ops/second
 - mkdir 173 (time 1782243576.18 total 10.01 last 17.27)
 - mkdir 328 (time 1782243586.22 total 20.06 last 15.43)
 - mkdir 476 (time 1782243596.32 total 30.16 last 14.65)
 - mkdir 641 (time 1782243606.36 total 40.19 last 16.45)
 - mkdir 811 (time 1782243616.37 total 50.21 last 16.98)
 - mkdir 954 (time 1782243626.49 total 60.32 last 14.14)
total: 1000 mkdir in 63.57 seconds: 15.73 ops/second
Time for file creation 72 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 11 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.44 seconds: 22.79 ops/second
 - mkdir 139 (time 1782244166.52 total 10.07 last 13.80)
 - mkdir 295 (time 1782244176.54 total 20.10 last 15.56)
 - mkdir 441 (time 1782244186.57 total 30.12 last 14.57)
 - mkdir 592 (time 1782244196.58 total 40.13 last 15.09)
 - mkdir 737 (time 1782244206.59 total 50.14 last 14.49)
 - mkdir 879 (time 1782244216.61 total 60.16 last 14.17)
total: 1000 mkdir in 68.01 seconds: 14.70 ops/second
 - mkdir 133 (time 1782244164.59 total 10.05 last 13.23)
 - mkdir 277 (time 1782244174.62 total 20.08 last 14.36)
 - mkdir 420 (time 1782244184.68 total 30.14 last 14.21)
 - mkdir 568 (time 1782244194.72 total 40.18 last 14.75)
 - mkdir 718 (time 1782244204.75 total 50.21 last 14.95)
 - mkdir 852 (time 1782244214.78 total 60.24 last 13.36)
 - mkdir 995 (time 1782244224.80 total 70.26 last 14.26)
total: 1000 mkdir in 70.98 seconds: 14.09 ops/second
 - mkdir 131 (time 1782244165.43 total 10.03 last 13.07)
 - mkdir 276 (time 1782244175.48 total 20.08 last 14.42)
 - mkdir 419 (time 1782244185.57 total 30.16 last 14.18)
 - mkdir 572 (time 1782244195.62 total 40.21 last 15.22)
 - mkdir 720 (time 1782244205.67 total 50.27 last 14.72)
 - mkdir 857 (time 1782244215.69 total 60.28 last 13.68)
 - mkdir 995 (time 1782244225.75 total 70.34 last 13.72)
total: 1000 mkdir in 71.04 seconds: 14.08 ops/second
 - mkdir 136 (time 1782244166.30 total 10.05 last 13.53)
 - mkdir 283 (time 1782244176.32 total 20.07 last 14.68)
 - mkdir 424 (time 1782244186.38 total 30.12 last 14.02)
 - mkdir 579 (time 1782244196.41 total 40.16 last 15.45)
 - mkdir 723 (time 1782244206.51 total 50.26 last 14.25)
 - mkdir 864 (time 1782244216.51 total 60.26 last 14.10)
 - mkdir 994 (time 1782244226.63 total 70.37 last 12.85)
total: 1000 mkdir in 71.12 seconds: 14.06 ops/second
 - mkdir 134 (time 1782244165.22 total 10.01 last 13.38)
 - mkdir 279 (time 1782244175.25 total 20.04 last 14.47)
 - mkdir 416 (time 1782244185.28 total 30.07 last 13.65)
 - mkdir 564 (time 1782244195.33 total 40.12 last 14.73)
 - mkdir 705 (time 1782244205.33 total 50.12 last 14.10)
 - mkdir 846 (time 1782244215.38 total 60.17 last 14.02)
 - mkdir 984 (time 1782244225.49 total 70.28 last 13.66)
total: 1000 mkdir in 72.47 seconds: 13.80 ops/second
 - mkdir 133 (time 1782244166.55 total 10.12 last 13.14)
 - mkdir 285 (time 1782244176.58 total 20.16 last 15.14)
 - mkdir 424 (time 1782244186.67 total 30.24 last 13.79)
 - mkdir 578 (time 1782244196.72 total 40.30 last 15.32)
 - mkdir 726 (time 1782244206.75 total 50.33 last 14.75)
 - mkdir 864 (time 1782244216.77 total 60.35 last 13.77)
 - mkdir 995 (time 1782244226.90 total 70.48 last 12.94)
total: 1000 mkdir in 71.27 seconds: 14.03 ops/second
 - mkdir 136 (time 1782244166.17 total 10.02 last 13.58)
 - mkdir 282 (time 1782244176.20 total 20.04 last 14.56)
 - mkdir 424 (time 1782244186.23 total 30.08 last 14.15)
 - mkdir 576 (time 1782244196.33 total 40.17 last 15.06)
 - mkdir 718 (time 1782244206.37 total 50.22 last 14.13)
 - mkdir 855 (time 1782244216.40 total 60.24 last 13.66)
 - mkdir 990 (time 1782244226.44 total 70.28 last 13.45)
total: 1000 mkdir in 71.54 seconds: 13.98 ops/second
 - mkdir 133 (time 1782244166.46 total 10.03 last 13.26)
 - mkdir 276 (time 1782244176.51 total 20.08 last 14.23)
 - mkdir 414 (time 1782244186.52 total 30.10 last 13.78)
 - mkdir 564 (time 1782244196.55 total 40.13 last 14.95)
 - mkdir 707 (time 1782244206.58 total 50.16 last 14.26)
 - mkdir 848 (time 1782244216.59 total 60.16 last 14.09)
 - mkdir 982 (time 1782244226.62 total 70.19 last 13.36)
total: 1000 mkdir in 72.67 seconds: 13.76 ops/second
 - mkdir 130 (time 1782244165.95 total 10.01 last 12.98)
 - mkdir 265 (time 1782244176.03 total 20.09 last 13.40)
 - mkdir 399 (time 1782244186.09 total 30.15 last 13.32)
 - mkdir 552 (time 1782244196.11 total 40.17 last 15.26)
 - mkdir 694 (time 1782244206.16 total 50.22 last 14.13)
 - mkdir 835 (time 1782244216.17 total 60.23 last 14.09)
 - mkdir 963 (time 1782244226.27 total 70.32 last 12.68)
total: 1000 mkdir in 75.53 seconds: 13.24 ops/second
 - mkdir 126 (time 1782244166.19 total 10.01 last 12.59)
 - mkdir 263 (time 1782244176.19 total 20.01 last 13.70)
 - mkdir 395 (time 1782244186.19 total 30.01 last 13.20)
 - mkdir 547 (time 1782244196.28 total 40.10 last 15.07)
 - mkdir 693 (time 1782244206.29 total 50.11 last 14.59)
 - mkdir 826 (time 1782244216.34 total 60.16 last 13.23)
 - mkdir 955 (time 1782244226.36 total 70.17 last 12.88)
total: 1000 mkdir in 75.83 seconds: 13.19 ops/second
Time for file creation 88 sec for 1 RPC sent at a time
 - unlinked 0 (time 1782244242 ; total 0 ; last 0)
total: 1000 unlinks in 32 seconds: 31.250000 unlinks/second
 - unlinked 0 (time 1782244243 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1782244243 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1782244243 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1782244243 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1782244244 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1782244244 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1782244244 ; total 1 ; last 1)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1782244244 ; total 0 ; last 0)
total: 1000 unlinks in 33 seconds: 30.303030 unlinks/second
 - unlinked 0 (time 1782244243 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
Time for file removal 47 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8