== sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 06:28:51 (1787135331)
osp.lustre-MDT0000-osp-MDT0001.rpc_stats
osp.lustre-MDT0001-osp-MDT0000.rpc_stats
total: 10 mkdir in 0.58 seconds: 17.25 ops/second
 - mkdir 147 (time 1787135351.92 total 10.01 last 14.69)
 - mkdir 317 (time 1787135361.98 total 20.06 last 16.91)
 - mkdir 520 (time 1787135371.99 total 30.07 last 20.29)
 - mkdir 709 (time 1787135382.00 total 40.09 last 18.86)
 - mkdir 865 (time 1787135392.01 total 50.09 last 15.59)
total: 1000 mkdir in 56.94 seconds: 17.56 ops/second
 - mkdir 145 (time 1787135352.00 total 10.05 last 14.43)
 - mkdir 315 (time 1787135362.00 total 20.05 last 16.99)
 - mkdir 518 (time 1787135372.02 total 30.07 last 20.27)
 - mkdir 712 (time 1787135382.05 total 40.10 last 19.34)
 - mkdir 863 (time 1787135392.05 total 50.10 last 15.09)
total: 1000 mkdir in 57.03 seconds: 17.53 ops/second
 - mkdir 147 (time 1787135351.86 total 10.06 last 14.61)
 - mkdir 318 (time 1787135361.86 total 20.06 last 17.09)
 - mkdir 518 (time 1787135371.90 total 30.10 last 19.93)
 - mkdir 703 (time 1787135381.94 total 40.14 last 18.43)
 - mkdir 858 (time 1787135391.98 total 50.18 last 15.43)
total: 1000 mkdir in 57.51 seconds: 17.39 ops/second
 - mkdir 145 (time 1787135352.18 total 10.09 last 14.37)
 - mkdir 310 (time 1787135362.22 total 20.14 last 16.42)
 - mkdir 506 (time 1787135372.31 total 30.22 last 19.44)
 - mkdir 702 (time 1787135382.33 total 40.24 last 19.56)
 - mkdir 858 (time 1787135392.36 total 50.28 last 15.55)
total: 1000 mkdir in 57.32 seconds: 17.45 ops/second
 - mkdir 150 (time 1787135352.67 total 10.03 last 14.95)
 - mkdir 317 (time 1787135362.68 total 20.04 last 16.69)
 - mkdir 523 (time 1787135372.75 total 30.11 last 20.46)
 - mkdir 716 (time 1787135382.75 total 40.12 last 19.29)
 - mkdir 866 (time 1787135392.76 total 50.12 last 14.99)
total: 1000 mkdir in 57.14 seconds: 17.50 ops/second
 - mkdir 145 (time 1787135352.10 total 10.09 last 14.37)
 - mkdir 312 (time 1787135362.11 total 20.10 last 16.68)
 - mkdir 514 (time 1787135372.15 total 30.14 last 20.12)
 - mkdir 704 (time 1787135382.19 total 40.18 last 18.92)
 - mkdir 853 (time 1787135392.22 total 50.21 last 14.86)
total: 1000 mkdir in 57.92 seconds: 17.27 ops/second
 - mkdir 141 (time 1787135351.56 total 10.02 last 14.07)
 - mkdir 295 (time 1787135361.57 total 20.02 last 15.39)
 - mkdir 495 (time 1787135371.61 total 30.07 last 19.90)
 - mkdir 684 (time 1787135381.62 total 40.07 last 18.90)
 - mkdir 842 (time 1787135391.64 total 50.09 last 15.76)
total: 1000 mkdir in 58.56 seconds: 17.08 ops/second
 - mkdir 146 (time 1787135352.50 total 10.02 last 14.57)
 - mkdir 317 (time 1787135362.52 total 20.04 last 17.07)
 - mkdir 513 (time 1787135372.52 total 30.05 last 19.59)
 - mkdir 708 (time 1787135382.57 total 40.09 last 19.41)
 - mkdir 858 (time 1787135392.59 total 50.11 last 14.98)
total: 1000 mkdir in 57.66 seconds: 17.34 ops/second
 - mkdir 144 (time 1787135352.62 total 10.06 last 14.32)
 - mkdir 317 (time 1787135362.65 total 20.08 last 17.25)
 - mkdir 517 (time 1787135372.66 total 30.09 last 19.98)
 - mkdir 708 (time 1787135382.67 total 40.10 last 19.09)
 - mkdir 858 (time 1787135392.67 total 50.10 last 15.00)
total: 1000 mkdir in 57.63 seconds: 17.35 ops/second
 - mkdir 145 (time 1787135352.28 total 10.04 last 14.44)
 - mkdir 312 (time 1787135362.31 total 20.07 last 16.64)
 - mkdir 509 (time 1787135372.37 total 30.13 last 19.59)
 - mkdir 696 (time 1787135382.38 total 40.15 last 18.67)
 - mkdir 844 (time 1787135392.42 total 50.18 last 14.75)
total: 1000 mkdir in 58.67 seconds: 17.05 ops/second
Time for file creation 69 sec for 8 parallel RPCs
rmdir(/mnt/lustre/d182b.sanity/2/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/1/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/8/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/0/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/9/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/7/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/3/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/6/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/4/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
rmdir(/mnt/lustre/d182b.sanity/5/d-0) error: No such file or directory
total: 0 unlinks in 0 seconds: -nan unlinks/second
Time for file removal 11 sec for 8 parallel RPCs
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=1
total: 10 mkdir in 0.39 seconds: 25.52 ops/second
 - mkdir 138 (time 1787135939.73 total 10.07 last 13.70)
 - mkdir 383 (time 1787135949.77 total 20.11 last 24.39)
 - mkdir 593 (time 1787135959.80 total 30.14 last 20.94)
 - mkdir 752 (time 1787135969.81 total 40.15 last 15.89)
 - mkdir 918 (time 1787135979.82 total 50.16 last 16.59)
total: 1000 mkdir in 55.42 seconds: 18.05 ops/second
 - mkdir 151 (time 1787135941.07 total 10.04 last 15.05)
 - mkdir 406 (time 1787135951.09 total 20.06 last 25.44)
 - mkdir 616 (time 1787135961.12 total 30.09 last 20.94)
 - mkdir 769 (time 1787135971.12 total 40.09 last 15.29)
 - mkdir 928 (time 1787135981.16 total 50.12 last 15.85)
total: 1000 mkdir in 55.01 seconds: 18.18 ops/second
 - mkdir 144 (time 1787135941.58 total 10.05 last 14.33)
 - mkdir 404 (time 1787135951.61 total 20.07 last 25.93)
 - mkdir 606 (time 1787135961.64 total 30.10 last 20.14)
 - mkdir 759 (time 1787135971.66 total 40.13 last 15.26)
 - mkdir 922 (time 1787135981.68 total 50.14 last 16.27)
total: 1000 mkdir in 55.58 seconds: 17.99 ops/second
 - mkdir 123 (time 1787135938.87 total 10.07 last 12.22)
 - mkdir 355 (time 1787135948.94 total 20.14 last 23.03)
 - mkdir 560 (time 1787135958.94 total 30.14 last 20.50)
 - mkdir 722 (time 1787135968.97 total 40.16 last 16.16)
 - mkdir 882 (time 1787135979.03 total 50.22 last 15.90)
total: 1000 mkdir in 58.72 seconds: 17.03 ops/second
 - mkdir 141 (time 1787135940.94 total 10.03 last 14.06)
 - mkdir 393 (time 1787135950.94 total 20.03 last 25.19)
 - mkdir 597 (time 1787135960.98 total 30.07 last 20.33)
 - mkdir 747 (time 1787135971.09 total 40.18 last 14.84)
 - mkdir 909 (time 1787135981.10 total 50.19 last 16.18)
total: 1000 mkdir in 57.13 seconds: 17.50 ops/second
 - mkdir 142 (time 1787135941.46 total 10.01 last 14.19)
 - mkdir 395 (time 1787135951.49 total 20.04 last 25.22)
 - mkdir 595 (time 1787135961.50 total 30.05 last 19.99)
 - mkdir 748 (time 1787135971.55 total 40.10 last 15.23)
 - mkdir 909 (time 1787135981.56 total 50.12 last 16.07)
total: 1000 mkdir in 57.04 seconds: 17.53 ops/second
 - mkdir 135 (time 1787135940.71 total 10.03 last 13.45)
 - mkdir 388 (time 1787135950.74 total 20.07 last 25.22)
 - mkdir 591 (time 1787135960.78 total 30.10 last 20.24)
 - mkdir 737 (time 1787135970.81 total 40.13 last 14.55)
 - mkdir 903 (time 1787135980.81 total 50.14 last 16.59)
total: 1000 mkdir in 58.08 seconds: 17.22 ops/second
 - mkdir 147 (time 1787135941.59 total 10.03 last 14.66)
 - mkdir 401 (time 1787135951.61 total 20.05 last 25.35)
 - mkdir 598 (time 1787135961.67 total 30.11 last 19.58)
 - mkdir 744 (time 1787135971.71 total 40.15 last 14.54)
 - mkdir 907 (time 1787135981.74 total 50.18 last 16.26)
total: 1000 mkdir in 57.37 seconds: 17.43 ops/second
 - mkdir 147 (time 1787135941.64 total 10.01 last 14.68)
 - mkdir 398 (time 1787135951.74 total 20.11 last 24.85)
 - mkdir 598 (time 1787135961.78 total 30.15 last 19.92)
 - mkdir 748 (time 1787135971.80 total 40.17 last 14.97)
 - mkdir 910 (time 1787135981.85 total 50.23 last 16.11)
total: 1000 mkdir in 57.60 seconds: 17.36 ops/second
 - mkdir 131 (time 1787135940.60 total 10.02 last 13.08)
 - mkdir 370 (time 1787135950.61 total 20.03 last 23.86)
 - mkdir 571 (time 1787135960.67 total 30.09 last 19.99)
 - mkdir 717 (time 1787135970.73 total 40.15 last 14.50)
 - mkdir 881 (time 1787135980.75 total 50.17 last 16.37)
total: 1000 mkdir in 59.73 seconds: 16.74 ops/second
Time for file creation 70 sec for 1 RPC sent at a time
 - unlinked 0 (time 1787135998 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1787135999 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1787136000 ; total 0 ; last 0)
total: 1000 unlinks in 34 seconds: 29.411764 unlinks/second
 - unlinked 0 (time 1787135999 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
 - unlinked 0 (time 1787135999 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787135999 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787135999 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787136000 ; total 1 ; last 1)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787135999 ; total 0 ; last 0)
total: 1000 unlinks in 36 seconds: 27.777779 unlinks/second
 - unlinked 0 (time 1787136000 ; total 0 ; last 0)
total: 1000 unlinks in 35 seconds: 28.571428 unlinks/second
Time for file removal 47 sec for 1 RPC sent at a time
osp.lustre-MDT0001-osp-MDT0000.max_mod_rpcs_in_flight=8