== conf-sanity test 73c: Server mount doesn't fail with > 32 nets ========================================================== 23:28:36 (1767328116) umount lustre on /mnt/lustre..... stop ost1 service on oleg405-server stop mds service on oleg405-server stop mds service on oleg405-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing unload_modules_local oleg405-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1053: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1054: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing load_modules_local oleg405-server: Loading modules from /home/green/git/lustre-release/lustre oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1053: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1054: echo: write error: Device or resource busy oleg405-server: detected 4 online CPUs by sysfs oleg405-server: MODOPTS_LIBCFS= oleg405-server: Force libcfs to create 2 CPU partitions oleg405-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg405-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory oleg405-server: sh: /usr/sbin/ksocklnd-config: No such file or directory restart with 35 nets start mds service on oleg405-server pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1 pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1 oleg405-server: losetup: /dev/mapper/mds1_flakey: failed to set up loop device: No such file or directory pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1 oleg405-server: e2label: No such file or directory while trying to open /dev/mapper/mds1_flakey oleg405-server: Couldn't find valid filesystem superblock. pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1 Start mds1: mount -t lustre -o localrecov,loop /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg405-server: mount: /mnt/lustre-mds1: failed to setup loop device for /dev/mapper/mds1_flakey. pdsh@oleg405-client: oleg405-server: ssh exited with exit code 32 Start of /dev/mapper/mds1_flakey on mds1 failed 32 start ost1 service on oleg405-server pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1 pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1 oleg405-server: losetup: /dev/mapper/ost1_flakey: failed to set up loop device: No such file or directory pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1 oleg405-server: e2label: No such file or directory while trying to open /dev/mapper/ost1_flakey oleg405-server: Couldn't find valid filesystem superblock. pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1 Start ost1: mount -t lustre -o localrecov,loop /dev/mapper/ost1_flakey /mnt/lustre-ost1 oleg405-server: mount: /mnt/lustre-ost1: failed to setup loop device for /dev/mapper/ost1_flakey. pdsh@oleg405-client: oleg405-server: ssh exited with exit code 32 pdsh@oleg405-client: oleg405-server: ssh exited with exit code 1 oleg405-server: error: set_param: param_path 'seq/cli--super/width': No such file or directory oleg405-server: error: set_param: setting 'seq/cli--super/width'='65536': No such file or directory pdsh@oleg405-client: oleg405-server: ssh exited with exit code 2 Start of /dev/mapper/ost1_flakey on ost1 failed 32 conf-sanity test_73c: @@@@@@ FAIL: unable to start ost1 Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7487:error() = /home/green/git/lustre-release/lustre/tests/conf-sanity.sh:6634:test_73c() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7863:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7926:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7726:run_test() = /home/green/git/lustre-release/lustre/tests/conf-sanity.sh:6636:main() Dumping lctl log to /tmp/testlogs//conf-sanity.test_73c.*.1767328290.log rsync: chown "/tmp/testlogs/.conf-sanity.test_73c.debug_log.oleg405-server.1767328290.log.kmzmj0" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.conf-sanity.test_73c.dmesg.oleg405-server.1767328290.log.Xew8bQ" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3] umount lustre on /mnt/lustre..... stop ost1 service on oleg405-server stop mds service on oleg405-server stop mds service on oleg405-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg405-server' oleg405-server: oleg405-server.virtnet: executing unload_modules_local oleg405-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg405-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded.