== replay-single test 70c: tar 2mdts recovery ============ 09:15:43 (1715346943) Starting client oleg203-client.virtnet: -o user_xattr,flock oleg203-server@tcp:/lustre /mnt/lustre Started clients oleg203-client.virtnet: 192.168.202.103@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt,statfs_project) Started tar 2499 striped dir -i0 -c2 -H crush2 /mnt/lustre/d70c.replay-single tar: Removing leading `/' from member names tar: Removing leading `/' from hard link targets tar: Removing leading `/' from member names tar: Removing leading `/' from hard link targets tar: Removing leading `/' from member names UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 6560 1281128 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 4396 1283292 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 17576 3575120 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 6108 3588224 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 23684 7163344 1% /mnt/lustre tar: Removing leading `/' from hard link targets test_70c fail mds2 1 times Failing mds2 on oleg203-server Stopping /mnt/lustre-mds2 (opts:) on oleg203-server 09:17:57 (1715347077) shut down Failover mds2 to oleg203-server mount facets: mds2 Starting mds2: -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg203-server: oleg203-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg203-client: oleg203-server: ssh exited with exit code 1 Started lustre-MDT0001 09:18:12 (1715347092) targets are mounted 09:18:12 (1715347092) facet_failover done oleg203-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0001-mdc-*.mds_server_uuid mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec tar: Removing leading `/' from member names tar: Removing leading `/' from hard link targets tar: Removing leading `/' from member names tar: Removing leading `/' from hard link targets UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 1414116 6856 1280832 1% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 1414116 4320 1283368 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 3833116 5848 3588692 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3833116 20252 3571048 1% /mnt/lustre[OST:1] filesystem_summary: 7666232 26100 7159740 1% /mnt/lustre test_70c fail mds1 2 times Failing mds1 on oleg203-server Stopping /mnt/lustre-mds1 (opts:) on oleg203-server 09:20:33 (1715347233) shut down Failover mds1 to oleg203-server mount facets: mds1 Starting mds1: -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg203-server: oleg203-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg203-client: oleg203-server: ssh exited with exit code 1 Started lustre-MDT0000 09:20:46 (1715347246) targets are mounted 09:20:46 (1715347246) facet_failover done oleg203-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec