== conf-sanity test 69: replace an OST with the same index ========================================================== 02:54:08 (1785394448) start mds service on oleg429-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg429-server' oleg429-server: oleg429-server.virtnet: executing load_modules_local oleg429-server: Loading modules from /home/green/git/lustre-release/lustre oleg429-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg429-server: detected 4 online CPUs by sysfs oleg429-server: MODOPTS_LIBCFS= oleg429-server: Force libcfs to create 2 CPU partitions oleg429-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg429-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg429-server: mdt/mdt options: 'mdt_enable_flr_ec=1' Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg429-server: oleg429-server.virtnet: executing set_default_debug -1 all pdsh@oleg429-client: oleg429-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg429-server Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg429-server: oleg429-server.virtnet: executing set_default_debug -1 all pdsh@oleg429-client: oleg429-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg429-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg429-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg429-server: oleg429-server.virtnet: executing set_default_debug -1 all pdsh@oleg429-client: oleg429-server: ssh exited with exit code 1 Started lustre-OST0000 oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg429-client.virtnet: -o user_xattr,flock 192.168.204.129@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 49503 inodes available. Want 99872. rc=0 - open/close 549 (time 1785394603.73 total 10.03 last 54.75) - open/close 1021 (time 1785394613.74 total 20.04 last 47.15) - open/close 1514 (time 1785394623.76 total 30.06 last 49.20) - open/close 2129 (time 1785394633.76 total 40.06 last 61.47) - open/close 2608 (time 1785394643.77 total 50.07 last 47.88) - open/close 2987 (time 1785394653.77 total 60.07 last 37.89) - open/close 3379 (time 1785394663.79 total 70.09 last 39.14) - open/close 3764 (time 1785394673.80 total 80.10 last 38.45) - open/close 4344 (time 1785394683.80 total 90.10 last 57.99) - open/close 5152 (time 1785394693.81 total 100.11 last 80.75) - open/close 5661 (time 1785394703.82 total 110.12 last 50.87) - open/close 6127 (time 1785394713.84 total 120.14 last 46.50) - open/close 6618 (time 1785394723.84 total 130.14 last 49.08) - open/close 7083 (time 1785394733.85 total 140.14 last 46.49) - open/close 7634 (time 1785394743.85 total 150.15 last 55.09) - open/close 8354 (time 1785394753.86 total 160.16 last 71.94) - open/close 8927 (time 1785394763.87 total 170.17 last 57.24) - open/close 9497 (time 1785394773.88 total 180.17 last 56.95) - open/close 9990 (time 1785394783.88 total 190.18 last 49.29) total: 10000 open/close in 190.40 seconds: 52.52 ops/second - unlinked 0 (time 1785394790 ; total 0 ; last 0) total: 10000 unlinks in 95 seconds: 105.263161 unlinks/second - open/close 490 (time 1785394900.48 total 10.01 last 48.98) - open/close 970 (time 1785394910.48 total 20.01 last 47.98) - open/close 1545 (time 1785394920.50 total 30.03 last 57.40) - open/close 2175 (time 1785394930.52 total 40.05 last 62.87) - open/close 2718 (time 1785394940.53 total 50.06 last 54.24) - open/close 3079 (time 1785394950.53 total 60.06 last 36.09) - open/close 3475 (time 1785394960.54 total 70.07 last 39.57) - open/close 3905 (time 1785394970.59 total 80.12 last 42.79) - open/close 4372 (time 1785394980.60 total 90.13 last 46.63) - open/close 4841 (time 1785394990.62 total 100.14 last 46.85) - open/close 5304 (time 1785395000.63 total 110.16 last 46.22) - open/close 5745 (time 1785395010.64 total 120.17 last 44.07) - open/close 6401 (time 1785395020.64 total 130.17 last 65.57) - open/close 7217 (time 1785395030.65 total 140.18 last 81.55) - open/close 7934 (time 1785395040.65 total 150.18 last 71.66) - open/close 8603 (time 1785395050.67 total 160.20 last 66.81) - open/close 9276 (time 1785395060.67 total 170.20 last 67.29) - open/close 9785 (time 1785395070.67 total 180.20 last 50.87) total: 10000 open/close in 185.86 seconds: 53.80 ops/second - unlinked 0 (time 1785395084 ; total 0 ; last 0) total: 10000 unlinks in 93 seconds: 107.526878 unlinks/second - open/close 489 (time 1785395194.47 total 10.03 last 48.77) - open/close 1061 (time 1785395204.49 total 20.05 last 57.07) - open/close 1617 (time 1785395214.50 total 30.06 last 55.53) - open/close 2137 (time 1785395224.51 total 40.07 last 51.98) - open/close 3130 (time 1785395234.51 total 50.07 last 99.25) - open/close 3640 (time 1785395244.54 total 60.10 last 50.87) - open/close 3982 (time 1785395254.57 total 70.13 last 34.10) - open/close 4494 (time 1785395264.58 total 80.14 last 51.13) - open/close 5017 (time 1785395274.60 total 90.16 last 52.21) - open/close 5508 (time 1785395284.60 total 100.16 last 49.08) - open/close 5950 (time 1785395294.64 total 110.20 last 44.05) - open/close 6307 (time 1785395304.64 total 120.20 last 35.70) - open/close 6682 (time 1785395314.65 total 130.20 last 37.47) - open/close 7182 (time 1785395324.65 total 140.20 last 50.00) - open/close 7741 (time 1785395334.66 total 150.22 last 55.82) - open/close 8292 (time 1785395344.67 total 160.23 last 55.06) - open/close 8771 (time 1785395354.67 total 170.23 last 47.89) - open/close 9274 (time 1785395364.69 total 180.25 last 50.19) - open/close 9693 (time 1785395374.71 total 190.27 last 41.84) total: 10000 open/close in 197.27 seconds: 50.69 ops/second - unlinked 0 (time 1785395388 ; total 0 ; last 0) total: 10000 unlinks in 112 seconds: 89.285713 unlinks/second - open/close 405 (time 1785395515.85 total 10.00 last 40.49) - open/close 891 (time 1785395525.86 total 20.01 last 48.57) - open/close 1394 (time 1785395535.87 total 30.01 last 50.28) - open/close 2013 (time 1785395545.87 total 40.01 last 61.89) - open/close 2449 (time 1785395555.89 total 50.04 last 43.48) - open/close 2984 (time 1785395565.91 total 60.06 last 53.43) - open/close 3697 (time 1785395575.91 total 70.06 last 71.30) - open/close 4345 (time 1785395585.91 total 80.06 last 64.77) - open/close 5082 (time 1785395595.92 total 90.07 last 73.67) - open/close 5911 (time 1785395605.92 total 100.07 last 82.87) - open/close 6345 (time 1785395615.93 total 110.07 last 43.38) - open/close 6925 (time 1785395625.94 total 120.09 last 57.89) - open/close 7580 (time 1785395635.97 total 130.12 last 65.35) - open/close 8464 (time 1785395645.97 total 140.12 last 88.38) - open/close 9289 (time 1785395655.99 total 150.14 last 82.34) - open/close 9869 (time 1785395666.00 total 160.14 last 57.96) total: 10000 open/close in 162.85 seconds: 61.41 ops/second - unlinked 0 (time 1785395675 ; total 0 ; last 0) total: 10000 unlinks in 92 seconds: 108.695656 unlinks/second - open/close 462 (time 1785395784.69 total 10.01 last 46.18) - open/close 1075 (time 1785395794.70 total 20.01 last 61.27) - open/close 1787 (time 1785395804.70 total 30.01 last 71.18) - open/close 2448 (time 1785395814.72 total 40.04 last 65.95) - open/close 2977 (time 1785395824.73 total 50.04 last 52.87) - open/close 3477 (time 1785395834.74 total 60.05 last 49.95) - open/close 4032 (time 1785395844.75 total 70.06 last 55.45) - open/close 4579 (time 1785395854.76 total 80.08 last 54.62) - open/close 5023 (time 1785395864.79 total 90.10 last 44.30) - open/close 5499 (time 1785395874.80 total 100.11 last 47.54) - open/close 6080 (time 1785395884.81 total 110.12 last 58.05) - open/close 6557 (time 1785395894.82 total 120.13 last 47.64) - open/close 6949 (time 1785395904.82 total 130.13 last 39.19) - open/close 7415 (time 1785395914.85 total 140.17 last 46.44) - open/close 7961 (time 1785395924.86 total 150.17 last 54.57) - open/close 8408 (time 1785395934.87 total 160.18 last 44.64) - open/close 8984 (time 1785395944.89 total 170.21 last 57.48) - open/close 9393 (time 1785395954.90 total 180.21 last 40.89) - open/close 9781 (time 1785395964.90 total 190.22 last 38.77) total: 10000 open/close in 193.84 seconds: 51.59 ops/second - unlinked 0 (time 1785395975 ; total 0 ; last 0) total: 10000 unlinks in 103 seconds: 97.087379 unlinks/second - open/close 470 (time 1785396093.99 total 10.02 last 46.92) - open/close 1263 (time 1785396103.99 total 20.02 last 79.30) - open/close 1813 (time 1785396114.01 total 30.04 last 54.89) - open/close 2475 (time 1785396124.02 total 40.04 last 66.18) - open/close 2961 (time 1785396134.04 total 50.06 last 48.51) - open/close 3788 (time 1785396144.04 total 60.06 last 82.69) - open/close 4452 (time 1785396154.05 total 70.07 last 66.31) - open/close 5182 (time 1785396164.08 total 80.11 last 72.77) - open/close 6006 (time 1785396174.09 total 90.11 last 82.38) - open/close 6568 (time 1785396184.09 total 100.12 last 56.16) - open/close 7227 (time 1785396194.10 total 110.13 last 65.84) - open/close 7703 (time 1785396204.11 total 120.13 last 47.57) - open/close 8252 (time 1785396214.14 total 130.17 last 54.71) - open/close 8755 (time 1785396224.16 total 140.18 last 50.24) - open/close 9467 (time 1785396234.17 total 150.19 last 71.11) total: 10000 open/close in 159.34 seconds: 62.76 ops/second - unlinked 0 (time 1785396249 ; total 0 ; last 0) total: 10000 unlinks in 115 seconds: 86.956520 unlinks/second - open/close 647 (time 1785396380.66 total 10.02 last 64.57) - open/close 1226 (time 1785396390.67 total 20.02 last 57.89) - open/close 1669 (time 1785396400.68 total 30.03 last 44.24) - open/close 2044 (time 1785396410.70 total 40.05 last 37.43) - open/close 2475 (time 1785396420.70 total 50.05 last 43.10) - open/close 2949 (time 1785396430.72 total 60.08 last 47.28) - open/close 3245 (time 1785396440.74 total 70.10 last 29.55) - open/close 3610 (time 1785396450.75 total 80.10 last 36.47) - open/close 4070 (time 1785396460.77 total 90.12 last 45.92) - open/close 4434 (time 1785396470.78 total 100.13 last 36.36) - open/close 5197 (time 1785396480.78 total 110.13 last 76.27) - open/close 5737 (time 1785396490.78 total 120.14 last 53.99) - open/close 6201 (time 1785396500.80 total 130.16 last 46.31) - open/close 6727 (time 1785396510.81 total 140.17 last 52.53) - open/close 7352 (time 1785396520.82 total 150.17 last 62.47) - open/close 7840 (time 1785396530.84 total 160.20 last 48.69) - open/close 8241 (time 1785396540.84 total 170.20 last 40.09) - open/close 8784 (time 1785396550.85 total 180.20 last 54.28) - open/close 9327 (time 1785396560.85 total 190.21 last 54.26) - open/close 9806 (time 1785396570.87 total 200.22 last 47.83) total: 10000 open/close in 204.18 seconds: 48.98 ops/second - unlinked 0 (time 1785396580 ; total 0 ; last 0) total: 10000 unlinks in 98 seconds: 102.040817 unlinks/second - open/close 601 (time 1785396694.59 total 10.00 last 60.08) - open/close 1218 (time 1785396704.61 total 20.03 last 61.56) - open/close 1794 (time 1785396714.62 total 30.03 last 57.56) - open/close 2405 (time 1785396724.62 total 40.03 last 61.09) - open/close 3018 (time 1785396734.63 total 50.04 last 61.27) - open/close 3520 (time 1785396744.64 total 60.05 last 50.17) - open/close 4014 (time 1785396754.69 total 70.10 last 49.15) - open/close 4415 (time 1785396764.71 total 80.12 last 40.02) - open/close 4762 (time 1785396774.72 total 90.13 last 34.64) - open/close 5207 (time 1785396784.75 total 100.16 last 44.41) - open/close 5792 (time 1785396794.75 total 110.16 last 58.50) - open/close 6173 (time 1785396804.75 total 120.17 last 38.07) - open/close 6541 (time 1785396814.76 total 130.17 last 36.79) - open/close 7068 (time 1785396824.76 total 140.17 last 52.66) - open/close 7484 (time 1785396834.80 total 150.21 last 41.44) - open/close 7997 (time 1785396844.81 total 160.22 last 51.24) - open/close 8541 (time 1785396854.82 total 170.23 last 54.38) - open/close 9066 (time 1785396864.84 total 180.25 last 52.41) - open/close 9486 (time 1785396874.84 total 190.25 last 42.00) total: 10000 open/close in 198.55 seconds: 50.36 ops/second - unlinked 0 (time 1785396888 ; total 0 ; last 0) total: 10000 unlinks in 106 seconds: 94.339622 unlinks/second - open/close 414 (time 1785397011.13 total 10.02 last 41.32) - open/close 902 (time 1785397021.13 total 20.02 last 48.77) - open/close 1492 (time 1785397031.13 total 30.02 last 59.00) - open/close 2012 (time 1785397041.16 total 40.05 last 51.87) - open/close 2558 (time 1785397051.17 total 50.06 last 54.55) - open/close 3191 (time 1785397061.17 total 60.06 last 63.28) - open/close 3776 (time 1785397071.18 total 70.08 last 58.42) - open/close 4345 (time 1785397081.20 total 80.09 last 56.81) - open/close 5114 (time 1785397091.21 total 90.10 last 76.81) - open/close 6108 (time 1785397101.25 total 100.14 last 99.04) - open/close 6691 (time 1785397111.25 total 110.14 last 58.29) - open/close 7318 (time 1785397121.27 total 120.16 last 62.59) - open/close 7807 (time 1785397131.27 total 130.16 last 48.89) - open/close 8421 (time 1785397141.27 total 140.16 last 61.39) - open/close 8817 (time 1785397151.27 total 150.16 last 39.59) - open/close 9560 (time 1785397161.29 total 160.18 last 74.19) total: 10000 open/close in 169.61 seconds: 58.96 ops/second - unlinked 0 (time 1785397176 ; total 0 ; last 0) total: 10000 unlinks in 102 seconds: 98.039215 unlinks/second - open/close 532 (time 1785397295.41 total 10.02 last 53.08) - open/close 960 (time 1785397305.45 total 20.06 last 42.65) - open/close 1454 (time 1785397315.45 total 30.06 last 49.40) - open/close 1930 (time 1785397325.45 total 40.06 last 47.60) - open/close 2359 (time 1785397335.47 total 50.08 last 42.79) - open/close 2839 (time 1785397345.48 total 60.09 last 47.96) - open/close 3304 (time 1785397355.49 total 70.10 last 46.46) - open/close 3747 (time 1785397365.50 total 80.11 last 44.25) - open/close 4279 (time 1785397375.51 total 90.12 last 53.17) - open/close 4862 (time 1785397385.53 total 100.14 last 58.15) - open/close 5448 (time 1785397395.53 total 110.14 last 58.59) - open/close 6006 (time 1785397405.56 total 120.17 last 55.65) - open/close 6557 (time 1785397415.57 total 130.18 last 55.03) - open/close 7076 (time 1785397425.60 total 140.21 last 51.77) - open/close 7663 (time 1785397435.62 total 150.23 last 58.59) - open/close 8205 (time 1785397445.62 total 160.23 last 54.18) - open/close 8861 (time 1785397455.64 total 170.25 last 65.47) - open/close 9408 (time 1785397465.64 total 180.25 last 54.70) total: 10000 open/close in 190.04 seconds: 52.62 ops/second - unlinked 0 (time 1785397482 ; total 0 ; last 0) total: 10000 unlinks in 87 seconds: 114.942528 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg429-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg429-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg429-server Permanent disk data: Device: /dev/mapper/ost1_flakey Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x42 (OST update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.204.129@tcp sys.timeout=20 device size = 4096MB formatting backing filesystem ldiskfs on /dev/mapper/ost1_flakey target name lustre-OST0000 kilobytes 200000 options -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 4096 -L lustre-OST0000 -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/mapper/ost1_flakey 200000k start ost1 service on oleg429-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg429-server: oleg429-server.virtnet: executing set_default_debug -1 all pdsh@oleg429-client: oleg429-server: ssh exited with exit code 1 Started lustre-OST0000 oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg429-server: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg429-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg429-server: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg429-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg429-client.virtnet: -o user_xattr,flock 192.168.204.129@tcp:/lustre /mnt/lustre On OST0, 20430 used inodes rc=0 pdsh@oleg429-client: oleg429-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg429-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg429-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg429-server Permanent disk data: Device: /dev/mapper/ost1_flakey Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.204.129@tcp sys.timeout=20 device size = 4096MB formatting backing filesystem ldiskfs on /dev/mapper/ost1_flakey target name lustre:OST0000 kilobytes 200000 options -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 4096 -L lustre:OST0000 -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/mapper/ost1_flakey 200000k checking for existing Lustre data: found Read previous values: Device: /dev/mapper/ost1_flakey Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.204.129@tcp sys.timeout=20 Permanent disk data: Device: /dev/mapper/ost1_flakey Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs Flags: 0x42 (OST update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.204.129@tcp sys.timeout=20 start ost1 service on oleg429-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg429-server: oleg429-server.virtnet: executing set_default_debug -1 all pdsh@oleg429-client: oleg429-server: ssh exited with exit code 1 Started lustre-OST0000 oleg429-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg429-server: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg429-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg429-server: oleg429-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg429-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg429-client.virtnet: -o user_xattr,flock 192.168.204.129@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg429-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg429-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg429-server stop mds service on oleg429-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg429-server stop mds service on oleg429-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg429-server unloading modules via unload_modules_local on: 'oleg429-server' oleg429-server: oleg429-server.virtnet: executing unload_modules_local oleg429-server: modules unloaded. pdsh@oleg429-client: oleg429-client: ssh exited with exit code 2 pdsh@oleg429-client: oleg429-server: ssh exited with exit code 2 pdsh@oleg429-client: oleg429-client: ssh exited with exit code 2 pdsh@oleg429-client: oleg429-server: ssh exited with exit code 2