== sanity-lfsck test 9a: LFSCK speed control (1) ========= 09:11:01 (1785935461) Checking servers environments Checking clients oleg412-client.virtnet environments /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg412-server' oleg412-server: oleg412-server.virtnet: executing load_modules_local oleg412-server: Loading modules from /home/green/git/lustre-release/lustre oleg412-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg412-server: detected 4 online CPUs by sysfs oleg412-server: MODOPTS_LIBCFS= oleg412-server: Force libcfs to create 2 CPU partitions Setup mgs, mdt, osts Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg412-server: mount.lustre: according to /etc/mtab /dev/mapper/mds1_flakey is already mounted on /mnt/lustre-mds1 pdsh@oleg412-client: oleg412-server: ssh exited with exit code 17 Start of /dev/mapper/mds1_flakey on mds1 failed 17 Start mds2: mount -t lustre -o localrecov /dev/mapper/mds2_flakey /mnt/lustre-mds2 oleg412-server: mount.lustre: according to /etc/mtab /dev/mapper/mds2_flakey is already mounted on /mnt/lustre-mds2 pdsh@oleg412-client: oleg412-server: ssh exited with exit code 17 Start of /dev/mapper/mds2_flakey on mds2 failed 17 Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 oleg412-server: mount.lustre: according to /etc/mtab /dev/mapper/ost1_flakey is already mounted on /mnt/lustre-ost1 pdsh@oleg412-client: oleg412-server: ssh exited with exit code 17 seq.cli-lustre-OST0000-super.width=65536 Start of /dev/mapper/ost1_flakey on ost1 failed 17 Start ost2: mount -t lustre -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 oleg412-server: mount.lustre: according to /etc/mtab /dev/mapper/ost2_flakey is already mounted on /mnt/lustre-ost2 pdsh@oleg412-client: oleg412-server: ssh exited with exit code 17 seq.cli-lustre-OST0001-super.width=65536 Start of /dev/mapper/ost2_flakey on ost2 failed 17 Starting client: oleg412-client.virtnet: -o user_xattr,flock 192.168.204.112@tcp:/lustre /mnt/lustre Starting client oleg412-client.virtnet: -o user_xattr,flock 192.168.204.112@tcp:/lustre /mnt/lustre Started clients oleg412-client.virtnet: 192.168.204.112@tcp:/lustre on /mnt/lustre type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff969b84605800.idle_timeout=debug osc.lustre-OST0001-osc-ffff969b84605800.idle_timeout=debug disable quota as required - open/close 417 (time 1785935564.78 total 10.04 last 41.53) - open/close 788 (time 1785935574.83 total 20.09 last 36.93) - open/close 1225 (time 1785935584.84 total 30.10 last 43.66) - open/close 1613 (time 1785935594.85 total 40.11 last 38.76) - open/close 2069 (time 1785935604.85 total 50.11 last 45.59) - open/close 2550 (time 1785935614.85 total 60.12 last 48.07) - open/close 3083 (time 1785935624.86 total 70.12 last 53.27) - open/close 3767 (time 1785935634.86 total 80.13 last 68.38) - open/close 4593 (time 1785935644.86 total 90.13 last 82.59) total: 5000 open/close in 98.71 seconds: 50.65 ops/second Started LFSCK on the device lustre-MDT0000: scrub layout sanity-lfsck test_9a: @@@@@@ FAIL: (5.2) Got speed 112, expected more than 112 Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7496:error() = /home/green/git/lustre-release/lustre/tests/sanity-lfsck.sh:1159:test_9a() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7873:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7936:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7750:run_test() = /home/green/git/lustre-release/lustre/tests/sanity-lfsck.sh:1181:main() Dumping lctl log to /tmp/testlogs//sanity-lfsck.test_9a.*.1785935688.log rsync: chown "/tmp/testlogs/.sanity-lfsck.test_9a.debug_log.oleg412-server.1785935688.log.e4KmxG" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.sanity-lfsck.test_9a.dmesg.oleg412-server.1785935688.log.FbkXQ4" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3]