== sanity-flr test 200: stress test ====================== 10:54:54 (1730822094)
Starting client: oleg222-client.virtnet:  -o user_xattr,flock oleg222-server@tcp:/lustre /mnt/lustre2
Starting client: oleg222-client.virtnet:  -o user_xattr,flock oleg222-server@tcp:/lustre /mnt/lustre3
fail_loc=0x1A03
fail_loc=0x1A03
lock to resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 730528 ..
Extending file size to 5946880 ..
Extending file size to 4779264 ..
done
Extending file size to 8255904 ..
Extending file size to 6531296 ..
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
Extending file size to 8331008 ..
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Extending file size to 8097856 ..
failed
Extending file size to 8691424 ..
Extending file size to 8473568 ..
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed
Extending file size to 9123712 ..
Extending file size to 9235520 ..
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e resync_start' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e resync_start' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e resync_start' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed
lock to resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e resync_start' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e resync_start' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed
lock to resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done
lock to resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e resync_start' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
lock to resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e resync_start' ..failed
lock to resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done
lock to resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e resync_start' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e resync_start' ..failed
lock to resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done
lock to resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed
lock to resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e resync_start' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
lock to resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed
lock to resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e resync_start' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e resync_start' ..failed
lock to resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e resync_start' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e resync_start' ..failed
lock to resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done
lock to resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done
lock to resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..done
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
lock to resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done
lock to resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e resync_start' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e resync_start' ..failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e delay_before_copy -d 1' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..done
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
lock to resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e resync_start' ..Extending file size to 9139776 ..
failed
resync file /mnt/lustre3/f200.sanity-flr with 'mirror_io resync -e resync_start' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..failed
resync file /mnt/lustre3/f200.sanity-flr with '/home/green/git/lustre-release/lustre/utils/lfs mirror resync' ..Waiting 23158 23159 23160 23162 23165
failed
192.168.202.122@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0
Stopping client oleg222-client.virtnet /mnt/lustre2 (opts:)
192.168.202.122@tcp:/lustre /mnt/lustre3 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0
Stopping client oleg222-client.virtnet /mnt/lustre3 (opts:)
lfs mirror: '/mnt/lustre/f200.sanity-flr' llapi_mirror_resync_many: Input/output error.
3
 sanity-flr test_200: @@@@@@ FAIL: checksum error for mirror 2 
  Trace dump:
  = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6427:error()
  = /home/green/git/lustre-release/lustre/tests/sanity-flr.sh:3166:test_200()
  = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6744:run_one()
  = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6791:run_one_logged()
  = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6632:run_test()
  = /home/green/git/lustre-release/lustre/tests/sanity-flr.sh:3171:main()
Dumping lctl log to /tmp/testlogs//sanity-flr.test_200.*.1730822157.log
rsync: chown "/tmp/testlogs/.sanity-flr.test_200.debug_log.oleg222-server.1730822157.log.2y8mNR" failed: Operation not permitted (1)
rsync: chown "/tmp/testlogs/.sanity-flr.test_200.dmesg.oleg222-server.1730822157.log.iic1tY" failed: Operation not permitted (1)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1651) [generator=3.1.2]