== conf-sanity test 157c: test allow_register timeout functionality ========================================================== 13:05:46 (1788800746) start mds service on oleg111-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' -rwxr-xr-x. 1 green green 1921056 Sep 7 12:38 /home/green/git/lustre-release/lustre/utils/mount.lustre -rwxr-xr-x. 1 root root 12 Jan 31 2019 /usr/sbin/mount.lustre mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre loading modules on: 'oleg111-server' oleg111-server: oleg111-server.virtnet: executing load_modules_local oleg111-server: kmemleak disabled oleg111-server: Loading modules from /home/green/git/lustre-release/lustre oleg111-server: detected 4 online CPUs by sysfs oleg111-server: MODOPTS_LIBCFS= oleg111-server: Force libcfs to create 2 CPU partitions oleg111-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg111-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg111-server: mdt/mdt options: 'mdt_enable_flr_ec=1' oleg111-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg111-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg111-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Started lustre-OST0000 oleg111-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg111-client.virtnet: -o user_xattr,flock 192.168.201.111@tcp:/lustre /mnt/lustre allow_register=unlimited pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Permanent disk data: Device: /tmp/lustre-ost1_2 Target: fs2510:OST0000 Index: 0 Lustre FS: fs2510 Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.201.111@tcp:192.168.201.111@tcp sys.timeout=20 formatting backing filesystem ldiskfs on /dev/loop0 target name fs2510:OST0000 kilobytes 200000 options -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 4096 -L fs2510:OST0000 -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop0 200000k /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1875: OST_INDICES: bad array subscript pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Start fs2ost: mount -t lustre -o localrecov /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost seq.cli-fs2510-OST0000-super.width=65536 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Commit the device label on /tmp/lustre-ost1_2 Started fs2510-OST0000 /mnt/lustre-fs2ost: 83.1 MiB (87142400 bytes) trimmed allow_register=60s obdfilter.fs2510-OST0000.recovery_status=status: INACTIVE Waiting for allow_register timeout to expire Waiting 66s for '1' Waiting 56s for '1' Waiting 46s for '1' Waiting 36s for '1' Waiting 26s for '1' Waiting 16s for '1' Updated after 59s: want '1' got '1' pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Permanent disk data: Device: /tmp/lustre-ost2_2 Target: fs2510:OST0001 Index: 1 Lustre FS: fs2510 Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.201.111@tcp:192.168.201.111@tcp sys.timeout=20 formatting backing filesystem ldiskfs on /dev/loop1 target name fs2510:OST0001 kilobytes 200000 options -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 4096 -L fs2510:OST0001 -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop1 200000k /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1875: OST_INDICES: bad array subscript pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Start fs3ost: mount -t lustre -o localrecov /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost oleg111-server: mount.lustre: mount /dev/mapper/fs3ost_flakey at /mnt/lustre-fs3ost failed: Permission denied pdsh@oleg111-client: oleg111-server: ssh exited with exit code 13 oleg111-server: error: set_param: param_path 'seq/cli-fs2510:OST0001-super/width': No such file or directory oleg111-server: error: set_param: setting 'seq/cli-fs2510:OST0001-super/width'='65536': No such file or directory pdsh@oleg111-client: oleg111-server: ssh exited with exit code 2 Start of /dev/mapper/fs3ost_flakey on fs3ost failed 13 allow_register=unlimited /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1875: OST_INDICES: bad array subscript Start fs3ost: mount -t lustre -o localrecov /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost seq.cli-fs2510-OST0001-super.width=65536 oleg111-server: oleg111-server.virtnet: executing set_default_debug -1 all pdsh@oleg111-client: oleg111-server: ssh exited with exit code 1 Commit the device label on /tmp/lustre-ost2_2 Started fs2510-OST0001 /mnt/lustre-fs3ost: 83.1 MiB (87146496 bytes) trimmed obdfilter.fs2510-OST0000.recovery_status=status: INACTIVE obdfilter.fs2510-OST0001.recovery_status=status: INACTIVE Cleaning up test 157c... Stopping /mnt/lustre-fs2ost (opts:-f) on oleg111-server Stopping /mnt/lustre-fs3ost (opts:-f) on oleg111-server umount lustre on /mnt/lustre..... Stopping client oleg111-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg111-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg111-server stop mds service on oleg111-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg111-server unloading modules via unload_modules_local on: 'oleg111-server' oleg111-server: oleg111-server.virtnet: executing unload_modules_local oleg111-server: kmemleak disabled modules unloaded.