== recovery-small test 155: failover after client remount ========================================================== 04:02:27 (1768554147)
Stopping client oleg245-client.virtnet /mnt/lustre (opts:)
Starting client: oleg245-client.virtnet:  -o user_xattr,flock 192.168.202.145@tcp:/lustre /mnt/lustre
Replay barrier on lustre-MDT0000
Failing mds1 on oleg245-server
Stopping /mnt/lustre-mds1 (opts:) on oleg245-server
04:02:50 (1768554170) shut down
facet: mds1 facet_host: oleg245-server facet_failover_host: oleg245-server
Failover mds1 to oleg245-server
mount facets: mds1
Start mds1: mount -t lustre -o localrecov  lustre-mdt1/mdt1 /mnt/lustre-mds1
oleg245-server: oleg245-server.virtnet: executing set_default_debug -1 all
pdsh@oleg245-client: oleg245-server: ssh exited with exit code 1
Started lustre-MDT0000
04:03:13 (1768554193) targets are mounted
04:03:13 (1768554193) facet_failover done
 recovery-small test_155: @@@@@@ FAIL: total 1813
drwxr-xr-x 3 root root   11776 Jan 16 03:23 d107.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 03:24 d108.recovery-small
drwxrwxrwx 2 root root   11776 Jan 16 02:06 d10c.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 03:36 d135.recovery-small
drwxr-xr-x 4 root root   11776 Jan 16 03:37 d137.recovery-small
drwxr-xr-x 2 root root 1417728 Jan 16 03:55 d144c.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:08 d14.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:10 d18a.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:11 d18b.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:11 d18c.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:15 d20a.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:16 d20b.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:25 d24a.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:26 d24b.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:29 d27.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:37 d50.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:44 d51.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 03:00 d52.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 03:14 d60.recovery-small
-rw-r--r-- 1 root root      12 Jan 16 02:06 f10d.recovery-small
-rwxrwxrwx 1 root root       0 Jan 16 03:29 f113.recovery-small
-rw-r--r-- 1 root root    4096 Jan 16 03:26 f115a.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 03:26 f115b.recovery-small
-rw-r--r-- 1 root root    4096 Jan 16 03:26 f115c.recovery-small
-rw-r--r-- 1 root root    4096 Jan 16 03:27 f115d.recovery-small
-rw-r--r-- 1 root root    4096 Jan 16 03:27 f115e.recovery-small
-rw-r--r-- 1 root root    4096 Jan 16 03:27 f115f.recovery-small
-rw-r--r-- 1 root root    4096 Jan 16 03:27 f115g.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 03:28 f120.recovery-small
-rw-r--r-- 1 root root     512 Jan 16 03:35 f131.recovery-small
-rw-r--r-- 1 root root    4096 Jan 16 03:35 f132.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 03:57 f147.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 04:02 f155.recovery-small
-rw-r--r-- 1 root root  102400 Jan 16 02:08 f16.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:15 f19c.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 01:59 f1.recovery-small
-rw-rw-rw- 1 root root       0 Jan 16 01:59 f1.recovery-small.2
-rw-r--r-- 1 root root       0 Jan 16 02:24 f23.recovery-small
-rw-r--r-- 1 root root   10240 Jan 16 02:01 f4.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 03:00 f53a.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 03:01 f53b.recovery-small
-rwxr-xr-x 1 root root       0 Jan 16 03:01 f53c.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 03:02 f54.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 03:02 f54.recovery-small.1
-rw-r--r-- 1 root root     512 Jan 16 03:10 f59.recovery-small
-rw-r--r-- 1 root root       1 Jan 16 03:16 f65.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 03:16 f66.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 02:03 f8.recovery-small != total 1813
drwxr-xr-x 3 root root   11776 Jan 16 03:23 d107.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 03:24 d108.recovery-small
drwxrwxrwx 2 root root   11776 Jan 16 02:06 d10c.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 03:36 d135.recovery-small
drwxr-xr-x 4 root root   11776 Jan 16 03:37 d137.recovery-small
drwxr-xr-x 2 root root 1417728 Jan 16 03:55 d144c.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:08 d14.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:10 d18a.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:11 d18b.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:11 d18c.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:15 d20a.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:16 d20b.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:25 d24a.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:26 d24b.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:29 d27.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:37 d50.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:44 d51.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 03:00 d52.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 03:14 d60.recovery-small
-rw-r--r-- 1 root root      12 Jan 16 02:06 f10d.recovery-small
-rwxrwxrwx 1 root root       0 Jan 16 03:29 f113.recovery-small
-rw-r--r-- 1 root root    4096 Jan 16 03:26 f115a.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 03:26 f115b.recovery-small
-rw-r--r-- 1 root root    4096 Jan 16 03:26 f115c.recovery-small
-rw-r--r-- 1 root root    4096 Jan 16 03:27 f115d.recovery-small
-rw-r--r-- 1 root root    4096 Jan 16 03:27 f115e.recovery-small
-rw-r--r-- 1 root root    4096 Jan 16 03:27 f115f.recovery-small
-rw-r--r-- 1 root root    4096 Jan 16 03:27 f115g.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 03:28 f120.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 03:35 f131.recovery-small
-rw-r--r-- 1 root root    4096 Jan 16 03:35 f132.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 03:57 f147.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 04:02 f155.recovery-small
-rw-r--r-- 1 root root  102400 Jan 16 02:08 f16.recovery-small
drwxr-xr-x 2 root root   11776 Jan 16 02:15 f19c.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 01:59 f1.recovery-small
-rw-rw-rw- 1 root root       0 Jan 16 01:59 f1.recovery-small.2
-rw-r--r-- 1 root root       0 Jan 16 02:24 f23.recovery-small
-rw-r--r-- 1 root root   10240 Jan 16 02:01 f4.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 03:00 f53a.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 03:01 f53b.recovery-small
-rwxr-xr-x 1 root root       0 Jan 16 03:01 f53c.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 03:02 f54.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 03:02 f54.recovery-small.1
-rw-r--r-- 1 root root     512 Jan 16 03:10 f59.recovery-small
-rw-r--r-- 1 root root       1 Jan 16 03:16 f65.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 03:16 f66.recovery-small
-rw-r--r-- 1 root root       0 Jan 16 02:03 f8.recovery-small 
  Trace dump:
  = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7317:error()
  = /home/green/git/lustre-release/lustre/tests/recovery-small.sh:3635:test_155()
  = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7693:run_one()
  = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7756:run_one_logged()
  = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7571:run_test()
  = /home/green/git/lustre-release/lustre/tests/recovery-small.sh:3637:main()
Dumping lctl log to /tmp/testlogs//recovery-small.test_155.*.1768554198.log
rsync: chown "/tmp/testlogs/.recovery-small.test_155.debug_log.oleg245-server.1768554198.log.UjlbYx" failed: Operation not permitted (1)
rsync: chown "/tmp/testlogs/.recovery-small.test_155.dmesg.oleg245-server.1768554198.log.tMo0Ge" failed: Operation not permitted (1)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3]