== conf-sanity test 69: replace an OST with the same index ========================================================== 11:46:02 (1777045562) start mds service on oleg139-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg139-server' oleg139-server: oleg139-server.virtnet: executing load_modules_local oleg139-server: Loading modules from /home/green/git/lustre-release/lustre oleg139-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy oleg139-server: detected 4 online CPUs by sysfs oleg139-server: MODOPTS_LIBCFS= oleg139-server: Force libcfs to create 2 CPU partitions oleg139-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg139-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg139-server: mdt/mdt options: 'mdt_enable_flr_ec=1' Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg139-server: oleg139-server.virtnet: executing set_default_debug -1 all pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg139-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg139-server: oleg139-server.virtnet: executing set_default_debug -1 all pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg139-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg139-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg139-server: oleg139-server.virtnet: executing set_default_debug -1 all pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 Started lustre-OST0000 oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg139-client.virtnet: -o user_xattr,flock 192.168.201.139@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 49504 inodes available. Want 99872. rc=0 - open/close 226 (time 1777045847.44 total 10.02 last 22.55) - open/close 442 (time 1777045857.49 total 20.07 last 21.50) - open/close 664 (time 1777045867.50 total 30.08 last 22.18) - open/close 873 (time 1777045877.52 total 40.10 last 20.86) - open/close 1085 (time 1777045887.53 total 50.11 last 21.17) - open/close 1298 (time 1777045897.53 total 60.12 last 21.29) - open/close 1523 (time 1777045907.56 total 70.15 last 22.44) - open/close 1758 (time 1777045917.62 total 80.20 last 23.37) - open/close 1980 (time 1777045927.65 total 90.23 last 22.14) - open/close 2175 (time 1777045937.66 total 100.24 last 19.48) - open/close 2394 (time 1777045947.68 total 110.27 last 21.84) - open/close 2576 (time 1777045957.70 total 120.29 last 18.16) - open/close 2765 (time 1777045967.74 total 130.32 last 18.84) - open/close 2955 (time 1777045977.77 total 140.36 last 18.94) - open/close 3144 (time 1777045987.79 total 150.37 last 18.87) - open/close 3330 (time 1777045997.84 total 160.43 last 18.50) - open/close 3506 (time 1777046007.91 total 170.50 last 17.48) - open/close 3691 (time 1777046017.93 total 180.52 last 18.46) - open/close 3887 (time 1777046027.95 total 190.54 last 19.56) - open/close 4080 (time 1777046037.98 total 200.56 last 19.25) - open/close 4281 (time 1777046048.02 total 210.60 last 20.02) - open/close 4486 (time 1777046058.04 total 220.63 last 20.45) - open/close 4681 (time 1777046068.08 total 230.66 last 19.44) - open/close 4870 (time 1777046078.08 total 240.67 last 18.89) - open/close 5057 (time 1777046088.11 total 250.70 last 18.64) - open/close 5252 (time 1777046098.12 total 260.71 last 19.48) - open/close 5463 (time 1777046108.21 total 270.79 last 20.92) - open/close 5662 (time 1777046118.22 total 280.80 last 19.89) - open/close 5852 (time 1777046128.26 total 290.85 last 18.91) - open/close 6043 (time 1777046138.28 total 300.87 last 19.06) - open/close 6234 (time 1777046148.32 total 310.91 last 19.02) - open/close 6420 (time 1777046158.36 total 320.94 last 18.53) - open/close 6622 (time 1777046168.38 total 330.97 last 20.15) - open/close 6831 (time 1777046178.39 total 340.97 last 20.89) - open/close 7019 (time 1777046188.45 total 351.04 last 18.69) - open/close 7195 (time 1777046198.48 total 361.06 last 17.55) - open/close 7403 (time 1777046208.49 total 371.08 last 20.77) - open/close 7604 (time 1777046218.56 total 381.14 last 19.98) - open/close 7772 (time 1777046228.57 total 391.16 last 16.77) - open/close 7957 (time 1777046238.64 total 401.22 last 18.38) - open/close 8131 (time 1777046248.66 total 411.24 last 17.36) - open/close 8297 (time 1777046258.71 total 421.29 last 16.51) - open/close 8471 (time 1777046268.75 total 431.34 last 17.32) - open/close 8655 (time 1777046278.85 total 441.43 last 18.23) - open/close 8875 (time 1777046288.89 total 451.48 last 21.90) - open/close 9072 (time 1777046298.92 total 461.50 last 19.65) - open/close 9281 (time 1777046308.92 total 471.51 last 20.89) - open/close 9479 (time 1777046318.95 total 481.54 last 19.75) - open/close 9678 (time 1777046328.99 total 491.58 last 19.82) - open/close 9891 (time 1777046339.03 total 501.62 last 21.21) total: 10000 open/close in 506.83 seconds: 19.73 ops/second - unlinked 0 (time 1777046359 ; total 0 ; last 0) total: 10000 unlinks in 266 seconds: 37.593987 unlinks/second - open/close 192 (time 1777046648.24 total 10.11 last 18.99) - open/close 366 (time 1777046658.26 total 20.13 last 17.36) - open/close 568 (time 1777046668.27 total 30.14 last 20.18) - open/close 773 (time 1777046678.27 total 40.15 last 20.49) - open/close 982 (time 1777046688.28 total 50.15 last 20.89) - open/close 1192 (time 1777046698.32 total 60.19 last 20.92) - open/close 1383 (time 1777046708.32 total 70.19 last 19.10) - open/close 1586 (time 1777046718.36 total 80.23 last 20.22) - open/close 1774 (time 1777046728.41 total 90.29 last 18.70) - open/close 1974 (time 1777046738.44 total 100.32 last 19.94) - open/close 2170 (time 1777046748.45 total 110.33 last 19.57) - open/close 2359 (time 1777046758.49 total 120.36 last 18.84) - open/close 2561 (time 1777046768.55 total 130.42 last 20.08) - open/close 2739 (time 1777046778.56 total 140.44 last 17.78) - open/close 2923 (time 1777046788.59 total 150.47 last 18.34) - open/close 3115 (time 1777046798.64 total 160.51 last 19.12) - open/close 3317 (time 1777046808.69 total 170.56 last 20.09) - open/close 3514 (time 1777046818.69 total 180.56 last 19.70) - open/close 3706 (time 1777046828.69 total 190.57 last 19.19) - open/close 3920 (time 1777046838.72 total 200.59 last 21.34) - open/close 4133 (time 1777046848.72 total 210.60 last 21.30) - open/close 4312 (time 1777046858.79 total 220.66 last 17.78) - open/close 4504 (time 1777046868.83 total 230.71 last 19.11) - open/close 4674 (time 1777046878.84 total 240.72 last 16.99) - open/close 4899 (time 1777046888.86 total 250.74 last 22.46) - open/close 5098 (time 1777046898.87 total 260.74 last 19.89) - open/close 5274 (time 1777046908.88 total 270.75 last 17.59) - open/close 5475 (time 1777046918.90 total 280.78 last 20.05) - open/close 5718 (time 1777046928.93 total 290.81 last 24.23) - open/close 5908 (time 1777046938.96 total 300.84 last 18.94) - open/close 6108 (time 1777046949.00 total 310.88 last 19.93) - open/close 6320 (time 1777046959.01 total 320.89 last 21.17) - open/close 6535 (time 1777046969.04 total 330.92 last 21.44) - open/close 6745 (time 1777046979.04 total 340.92 last 21.00) - open/close 6967 (time 1777046989.05 total 350.93 last 22.17) - open/close 7158 (time 1777046999.08 total 360.96 last 19.04) - open/close 7358 (time 1777047009.10 total 370.97 last 19.98) - open/close 7573 (time 1777047019.15 total 381.02 last 21.39) - open/close 7783 (time 1777047029.18 total 391.06 last 20.93) - open/close 7963 (time 1777047039.24 total 401.12 last 17.89) - open/close 8150 (time 1777047049.26 total 411.14 last 18.67) - open/close 8349 (time 1777047059.29 total 421.16 last 19.86) - open/close 8546 (time 1777047069.31 total 431.18 last 19.65) - open/close 8745 (time 1777047079.33 total 441.20 last 19.86) - open/close 8943 (time 1777047089.35 total 451.22 last 19.76) - open/close 9146 (time 1777047099.37 total 461.25 last 20.25) - open/close 9332 (time 1777047109.43 total 471.30 last 18.50) - open/close 9517 (time 1777047119.45 total 481.32 last 18.46) - open/close 9716 (time 1777047129.45 total 491.33 last 19.90) - open/close 9933 (time 1777047139.45 total 501.33 last 21.69) total: 10000 open/close in 505.08 seconds: 19.80 ops/second - unlinked 0 (time 1777047156 ; total 0 ; last 0) total: 10000 unlinks in 252 seconds: 39.682541 unlinks/second - open/close 277 (time 1777047429.44 total 10.03 last 27.63) - open/close 537 (time 1777047439.45 total 20.03 last 25.98) - open/close 771 (time 1777047449.46 total 30.04 last 23.38) - open/close 993 (time 1777047459.50 total 40.08 last 22.11) - open/close 1211 (time 1777047469.51 total 50.09 last 21.77) - open/close 1423 (time 1777047479.51 total 60.10 last 21.20) - open/close 1648 (time 1777047489.56 total 70.14 last 22.39) - open/close 1898 (time 1777047499.57 total 80.15 last 24.99) - open/close 2147 (time 1777047509.57 total 90.15 last 24.89) - open/close 2390 (time 1777047519.57 total 100.16 last 24.29) - open/close 2605 (time 1777047529.64 total 110.22 last 21.36) - open/close 2848 (time 1777047539.65 total 120.23 last 24.28) - open/close 3063 (time 1777047549.68 total 130.26 last 21.44) - open/close 3290 (time 1777047559.69 total 140.27 last 22.67) - open/close 3533 (time 1777047569.69 total 150.27 last 24.30) - open/close 3764 (time 1777047579.73 total 160.32 last 23.00) - open/close 3988 (time 1777047589.77 total 170.35 last 22.33) - open/close 4262 (time 1777047599.77 total 180.35 last 27.39) - open/close 4516 (time 1777047609.79 total 190.37 last 25.35) - open/close 4801 (time 1777047619.81 total 200.39 last 28.44) - open/close 5047 (time 1777047629.84 total 210.43 last 24.51) - open/close 5313 (time 1777047639.87 total 220.46 last 26.52) - open/close 5556 (time 1777047649.97 total 230.55 last 24.07) - open/close 5852 (time 1777047659.98 total 240.56 last 29.58) - open/close 6193 (time 1777047669.99 total 250.57 last 34.05) - open/close 6469 (time 1777047680.03 total 260.61 last 27.51) - open/close 6766 (time 1777047690.04 total 270.63 last 29.64) - open/close 7084 (time 1777047700.08 total 280.66 last 31.69) - open/close 7397 (time 1777047710.13 total 290.71 last 31.13) - open/close 7737 (time 1777047720.16 total 300.74 last 33.91) - open/close 8014 (time 1777047730.18 total 310.76 last 27.63) - open/close 8255 (time 1777047740.18 total 320.76 last 24.10) - open/close 8527 (time 1777047750.19 total 330.77 last 27.19) - open/close 8758 (time 1777047760.25 total 340.84 last 22.95) - open/close 9010 (time 1777047770.26 total 350.84 last 25.18) - open/close 9256 (time 1777047780.27 total 360.85 last 24.58) - open/close 9543 (time 1777047790.31 total 370.90 last 28.58) - open/close 9804 (time 1777047800.35 total 380.93 last 26.00) total: 10000 open/close in 388.64 seconds: 25.73 ops/second - unlinked 0 (time 1777047818 ; total 0 ; last 0) total: 10000 unlinks in 199 seconds: 50.251255 unlinks/second - open/close 255 (time 1777048035.18 total 10.01 last 25.47) - open/close 531 (time 1777048045.18 total 20.01 last 27.60) - open/close 818 (time 1777048055.19 total 30.02 last 28.67) - open/close 1138 (time 1777048065.20 total 40.04 last 31.95) - open/close 1449 (time 1777048075.22 total 50.05 last 31.07) - open/close 1850 (time 1777048085.23 total 60.06 last 40.05) - open/close 2205 (time 1777048095.25 total 70.08 last 35.43) - open/close 2560 (time 1777048105.26 total 80.09 last 35.45) - open/close 2915 (time 1777048115.26 total 90.09 last 35.50) - open/close 3340 (time 1777048125.28 total 100.12 last 42.41) - open/close 3685 (time 1777048135.30 total 110.14 last 34.43) - open/close 3996 (time 1777048145.31 total 120.14 last 31.08) - open/close 4373 (time 1777048155.33 total 130.17 last 37.62) - open/close 4766 (time 1777048165.35 total 140.18 last 39.25) - open/close 5144 (time 1777048175.35 total 150.18 last 37.79) - open/close 5513 (time 1777048185.35 total 160.18 last 36.90) - open/close 5877 (time 1777048195.37 total 170.20 last 36.33) - open/close 6250 (time 1777048205.38 total 180.21 last 37.26) - open/close 6573 (time 1777048215.39 total 190.22 last 32.28) - open/close 7020 (time 1777048225.39 total 200.22 last 44.67) - open/close 7358 (time 1777048235.41 total 210.24 last 33.75) - open/close 7718 (time 1777048245.42 total 220.25 last 35.95) - open/close 8108 (time 1777048255.43 total 230.26 last 38.95) - open/close 8543 (time 1777048265.44 total 240.27 last 43.47) - open/close 8983 (time 1777048275.46 total 250.29 last 43.90) - open/close 9325 (time 1777048285.47 total 260.30 last 34.18) - open/close 9662 (time 1777048295.49 total 270.32 last 33.64) total: 10000 open/close in 278.94 seconds: 35.85 ops/second - unlinked 0 (time 1777048311 ; total 0 ; last 0) total: 10000 unlinks in 165 seconds: 60.606060 unlinks/second - open/close 411 (time 1777048493.07 total 10.00 last 41.09) - open/close 819 (time 1777048503.10 total 20.03 last 40.67) - open/close 1229 (time 1777048513.11 total 30.04 last 40.96) - open/close 1602 (time 1777048523.16 total 40.09 last 37.12) - open/close 1984 (time 1777048533.17 total 50.10 last 38.18) - open/close 2359 (time 1777048543.19 total 60.11 last 37.43) - open/close 2709 (time 1777048553.19 total 70.11 last 35.00) - open/close 3103 (time 1777048563.19 total 80.12 last 39.37) - open/close 3427 (time 1777048573.21 total 90.14 last 32.34) - open/close 3765 (time 1777048583.23 total 100.16 last 33.73) - open/close 4109 (time 1777048593.25 total 110.18 last 34.33) - open/close 4440 (time 1777048603.27 total 120.20 last 33.05) - open/close 4785 (time 1777048613.28 total 130.21 last 34.45) - open/close 5144 (time 1777048623.29 total 140.22 last 35.88) - open/close 5485 (time 1777048633.32 total 150.25 last 33.98) - open/close 5886 (time 1777048643.33 total 160.26 last 40.06) - open/close 6292 (time 1777048653.34 total 170.27 last 40.57) - open/close 6699 (time 1777048663.35 total 180.28 last 40.67) - open/close 7113 (time 1777048673.39 total 190.32 last 41.23) - open/close 7504 (time 1777048683.42 total 200.35 last 38.98) - open/close 7850 (time 1777048693.43 total 210.36 last 34.55) - open/close 8212 (time 1777048703.44 total 220.37 last 36.17) - open/close 8625 (time 1777048713.46 total 230.39 last 41.23) - open/close 9093 (time 1777048723.47 total 240.39 last 46.77) - open/close 9544 (time 1777048733.49 total 250.42 last 45.01) total: 10000 open/close in 259.33 seconds: 38.56 ops/second - unlinked 0 (time 1777048748 ; total 0 ; last 0) total: 10000 unlinks in 124 seconds: 80.645164 unlinks/second - open/close 393 (time 1777048888.59 total 10.01 last 39.28) - open/close 755 (time 1777048898.62 total 20.03 last 36.10) - open/close 1146 (time 1777048908.62 total 30.03 last 39.09) - open/close 1545 (time 1777048918.63 total 40.04 last 39.88) - open/close 1946 (time 1777048928.63 total 50.04 last 40.09) - open/close 2362 (time 1777048938.63 total 60.04 last 41.60) - open/close 2754 (time 1777048948.63 total 70.05 last 39.18) - open/close 3228 (time 1777048958.67 total 80.08 last 47.24) - open/close 3703 (time 1777048968.67 total 90.09 last 47.47) - open/close 4181 (time 1777048978.68 total 100.10 last 47.76) - open/close 4635 (time 1777048988.71 total 110.12 last 45.30) - open/close 4995 (time 1777048998.73 total 120.14 last 35.91) - open/close 5432 (time 1777049008.75 total 130.16 last 43.62) - open/close 5851 (time 1777049018.77 total 140.18 last 41.84) - open/close 6335 (time 1777049028.77 total 150.18 last 48.37) - open/close 6895 (time 1777049038.78 total 160.20 last 55.94) - open/close 7467 (time 1777049048.79 total 170.21 last 57.13) - open/close 7966 (time 1777049058.81 total 180.22 last 49.81) - open/close 8651 (time 1777049068.82 total 190.23 last 68.45) - open/close 9215 (time 1777049078.82 total 200.23 last 56.39) - open/close 9869 (time 1777049088.84 total 210.25 last 65.29) total: 10000 open/close in 212.33 seconds: 47.10 ops/second - unlinked 0 (time 1777049095 ; total 0 ; last 0) total: 10000 unlinks in 88 seconds: 113.636360 unlinks/second - open/close 626 (time 1777049198.04 total 10.00 last 62.59) - open/close 1276 (time 1777049208.04 total 20.00 last 65.00) - open/close 1884 (time 1777049218.05 total 30.02 last 60.72) - open/close 2516 (time 1777049228.06 total 40.02 last 63.16) - open/close 3087 (time 1777049238.06 total 50.03 last 57.08) - open/close 3731 (time 1777049248.09 total 60.05 last 64.25) - open/close 4331 (time 1777049258.10 total 70.06 last 59.94) - open/close 4929 (time 1777049268.12 total 80.08 last 59.69) - open/close 5470 (time 1777049278.12 total 90.09 last 54.06) - open/close 5963 (time 1777049288.13 total 100.09 last 49.29) - open/close 6669 (time 1777049298.14 total 110.10 last 70.51) - open/close 7359 (time 1777049308.15 total 120.11 last 68.92) - open/close 7997 (time 1777049318.15 total 130.12 last 63.77) - open/close 8757 (time 1777049328.16 total 140.12 last 75.96) - open/close 9561 (time 1777049338.16 total 150.12 last 80.39) total: 10000 open/close in 156.65 seconds: 63.83 ops/second - unlinked 0 (time 1777049349 ; total 0 ; last 0) total: 10000 unlinks in 79 seconds: 126.582275 unlinks/second - open/close 660 (time 1777049442.64 total 10.00 last 65.97) - open/close 1409 (time 1777049452.64 total 20.01 last 74.87) - open/close 2000 (time 1777049462.65 total 30.02 last 59.06) - open/close 2536 (time 1777049472.66 total 40.02 last 53.57) - open/close 3226 (time 1777049482.66 total 50.02 last 68.99) - open/close 3871 (time 1777049492.66 total 60.03 last 64.46) - open/close 4484 (time 1777049502.67 total 70.03 last 61.28) - open/close 5106 (time 1777049512.68 total 80.04 last 62.14) - open/close 5804 (time 1777049522.68 total 90.05 last 69.76) - open/close 6543 (time 1777049532.69 total 100.05 last 73.86) - open/close 7273 (time 1777049542.70 total 110.06 last 72.94) - open/close 8082 (time 1777049552.70 total 120.07 last 80.82) - open/close 8828 (time 1777049562.71 total 130.07 last 74.57) - open/close 9625 (time 1777049572.71 total 140.08 last 79.65) total: 10000 open/close in 145.62 seconds: 68.67 ops/second - unlinked 0 (time 1777049581 ; total 0 ; last 0) total: 10000 unlinks in 68 seconds: 147.058823 unlinks/second - open/close 883 (time 1777049663.03 total 10.00 last 88.29) - open/close 1575 (time 1777049673.04 total 20.01 last 69.17) - open/close 2288 (time 1777049683.05 total 30.02 last 71.22) - open/close 3083 (time 1777049693.05 total 40.02 last 79.47) - open/close 3867 (time 1777049703.06 total 50.03 last 78.29) - open/close 4510 (time 1777049713.07 total 60.04 last 64.26) - open/close 5077 (time 1777049723.08 total 70.05 last 56.65) - open/close 5706 (time 1777049733.09 total 80.06 last 62.84) - open/close 6324 (time 1777049743.09 total 90.06 last 61.79) - open/close 6958 (time 1777049753.11 total 100.07 last 63.31) - open/close 7564 (time 1777049763.11 total 110.08 last 60.57) - open/close 8250 (time 1777049773.11 total 120.08 last 68.57) - open/close 9009 (time 1777049783.12 total 130.09 last 75.85) - open/close 9686 (time 1777049793.12 total 140.09 last 67.69) total: 10000 open/close in 144.52 seconds: 69.20 ops/second - unlinked 0 (time 1777049801 ; total 0 ; last 0) total: 10000 unlinks in 72 seconds: 138.888885 unlinks/second - open/close 766 (time 1777049887.23 total 10.01 last 76.52) - open/close 1582 (time 1777049897.24 total 20.02 last 81.54) - open/close 2352 (time 1777049907.24 total 30.02 last 76.97) - open/close 3136 (time 1777049917.24 total 40.02 last 78.40) - open/close 3862 (time 1777049927.25 total 50.03 last 72.56) - open/close 4596 (time 1777049937.25 total 60.03 last 73.35) - open/close 5257 (time 1777049947.27 total 70.05 last 65.99) - open/close 5964 (time 1777049957.27 total 80.05 last 70.67) - open/close 6656 (time 1777049967.28 total 90.06 last 69.16) - open/close 7297 (time 1777049977.28 total 100.06 last 64.09) - open/close 7989 (time 1777049987.28 total 110.06 last 69.19) - open/close 8638 (time 1777049997.29 total 120.07 last 64.86) - open/close 9392 (time 1777050007.30 total 130.08 last 75.32) total: 10000 open/close in 139.34 seconds: 71.76 ops/second - unlinked 0 (time 1777050020 ; total 0 ; last 0) total: 10000 unlinks in 66 seconds: 151.515152 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg139-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg139-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg139-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x42 (OST update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.201.139@tcp sys.timeout=20 device size = 4096MB formatting backing filesystem ldiskfs on /dev/mapper/ost1_flakey target name lustre-OST0000 kilobytes 200000 options -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 4096 -L lustre-OST0000 -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/mapper/ost1_flakey 200000k start ost1 service on oleg139-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg139-server: oleg139-server.virtnet: executing set_default_debug -1 all pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 Started lustre-OST0000 oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg139-server: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg139-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg139-server: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg139-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg139-client.virtnet: -o user_xattr,flock 192.168.201.139@tcp:/lustre /mnt/lustre On OST0, 20429 used inodes rc=0 pdsh@oleg139-client: oleg139-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg139-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg139-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg139-server Permanent disk data: Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.201.139@tcp sys.timeout=20 device size = 4096MB formatting backing filesystem ldiskfs on /dev/mapper/ost1_flakey target name lustre:OST0000 kilobytes 200000 options -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 4096 -L lustre:OST0000 -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/mapper/ost1_flakey 200000k checking for existing Lustre data: found Read previous values: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.201.139@tcp sys.timeout=20 Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x42 (OST update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.201.139@tcp sys.timeout=20 start ost1 service on oleg139-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg139-server: oleg139-server.virtnet: executing set_default_debug -1 all pdsh@oleg139-client: oleg139-server: ssh exited with exit code 1 Started lustre-OST0000 oleg139-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg139-server: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg139-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg139-server: oleg139-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg139-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg139-client.virtnet: -o user_xattr,flock 192.168.201.139@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg139-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg139-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg139-server stop mds service on oleg139-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg139-server stop mds service on oleg139-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg139-server unloading modules via unload_modules_local on: 'oleg139-server' oleg139-server: oleg139-server.virtnet: executing unload_modules_local oleg139-server: modules unloaded. pdsh@oleg139-client: oleg139-client: ssh exited with exit code 2 pdsh@oleg139-client: oleg139-server: ssh exited with exit code 2 pdsh@oleg139-client: oleg139-server: ssh exited with exit code 2 pdsh@oleg139-client: oleg139-client: ssh exited with exit code 2