== conf-sanity test 157c: test allow_register timeout functionality ========================================================== 13:08:45 (1788800925) start mds service on oleg602-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' -rwxr-xr-x. 1 green green 1921056 Sep 7 12:38 /home/green/git/lustre-release/lustre/utils/mount.lustre -rwxr-xr-x. 1 root root 12 Jan 31 2019 /usr/sbin/mount.lustre mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre loading modules on: 'oleg602-server' oleg602-server: oleg602-server.virtnet: executing load_modules_local oleg602-server: kmemleak disabled oleg602-server: Loading modules from /home/green/git/lustre-release/lustre oleg602-server: detected 4 online CPUs by sysfs oleg602-server: MODOPTS_LIBCFS= oleg602-server: Force libcfs to create 2 CPU partitions oleg602-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg602-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg602-server: mdt/mdt options: 'mdt_enable_flr_ec=1' oleg602-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Start mds1: mount -t lustre -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg602-server: oleg602-server.virtnet: executing set_default_debug -1 all pdsh@oleg602-client: oleg602-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg602-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg602-server Start ost1: mount -t lustre -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg602-server: oleg602-server.virtnet: executing set_default_debug -1 all pdsh@oleg602-client: oleg602-server: ssh exited with exit code 1 Started lustre-OST0000 oleg602-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg602-client.virtnet: -o user_xattr,flock 192.168.206.102@tcp:/lustre /mnt/lustre allow_register=unlimited pdsh@oleg602-client: oleg602-server: ssh exited with exit code 1 Permanent disk data: Device: /tmp/lustre-ost1_2 Target: fs25219:OST0000 Index: 0 Lustre FS: fs25219 Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.206.102@tcp:192.168.206.102@tcp sys.timeout=20 formatting backing filesystem ldiskfs on /dev/loop0 target name fs25219:OST0000 kilobytes 200000 options -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 4096 -L fs25219:OST0000 -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop0 200000k /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1875: OST_INDICES: bad array subscript pdsh@oleg602-client: oleg602-server: ssh exited with exit code 1 pdsh@oleg602-client: oleg602-server: ssh exited with exit code 1 Start fs2ost: mount -t lustre -o localrecov /dev/mapper/fs2ost_flakey /mnt/lustre-fs2ost seq.cli-fs25219-OST0000-super.width=65536 oleg602-server: oleg602-server.virtnet: executing set_default_debug -1 all pdsh@oleg602-client: oleg602-server: ssh exited with exit code 1 Commit the device label on /tmp/lustre-ost1_2 Started fs25219-OST0000 /mnt/lustre-fs2ost: 83.1 MiB (87142400 bytes) trimmed allow_register=60s obdfilter.fs25219-OST0000.recovery_status=status: INACTIVE Waiting for allow_register timeout to expire Waiting 67s for '1' Waiting 57s for '1' Waiting 47s for '1' Waiting 37s for '1' Waiting 27s for '1' Waiting 17s for '1' Updated after 61s: want '1' got '1' pdsh@oleg602-client: oleg602-server: ssh exited with exit code 1 Permanent disk data: Device: /tmp/lustre-ost2_2 Target: fs25219:OST0001 Index: 1 Lustre FS: fs25219 Mount type: ldiskfs Flags: 0x62 (OST first_time update ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=192.168.206.102@tcp:192.168.206.102@tcp sys.timeout=20 formatting backing filesystem ldiskfs on /dev/loop1 target name fs25219:OST0001 kilobytes 200000 options -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F mkfs_cmd = mke2fs -j -b 4096 -L fs25219:OST0001 -b 4096 -I 512 -q -O uninit_bg,extents,dir_nlink,quota,project,huge_file,large_dir,^fast_commit,flex_bg -G 256 -E lazy_itable_init,resize="4290772992",lazy_journal_init,packed_meta_blocks -F /dev/loop1 200000k /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1875: OST_INDICES: bad array subscript pdsh@oleg602-client: oleg602-server: ssh exited with exit code 1 pdsh@oleg602-client: oleg602-server: ssh exited with exit code 1 Start fs3ost: mount -t lustre -o localrecov /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost oleg602-server: mount.lustre: mount /dev/mapper/fs3ost_flakey at /mnt/lustre-fs3ost failed: Permission denied pdsh@oleg602-client: oleg602-server: ssh exited with exit code 13 oleg602-server: error: set_param: param_path 'seq/cli-fs25219:OST0001-super/width': No such file or directory oleg602-server: error: set_param: setting 'seq/cli-fs25219:OST0001-super/width'='65536': No such file or directory pdsh@oleg602-client: oleg602-server: ssh exited with exit code 2 Start of /dev/mapper/fs3ost_flakey on fs3ost failed 13 allow_register=unlimited /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1875: OST_INDICES: bad array subscript Start fs3ost: mount -t lustre -o localrecov /dev/mapper/fs3ost_flakey /mnt/lustre-fs3ost seq.cli-fs25219-OST0001-super.width=65536 oleg602-server: oleg602-server.virtnet: executing set_default_debug -1 all pdsh@oleg602-client: oleg602-server: ssh exited with exit code 1 Commit the device label on /tmp/lustre-ost2_2 Started fs25219-OST0001 /mnt/lustre-fs3ost: 83.1 MiB (87146496 bytes) trimmed obdfilter.fs25219-OST0000.recovery_status=status: INACTIVE obdfilter.fs25219-OST0001.recovery_status=status: INACTIVE Cleaning up test 157c... Stopping /mnt/lustre-fs2ost (opts:-f) on oleg602-server Stopping /mnt/lustre-fs3ost (opts:-f) on oleg602-server umount lustre on /mnt/lustre..... Stopping client oleg602-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg602-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg602-server stop mds service on oleg602-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg602-server unloading modules via unload_modules_local on: 'oleg602-server' oleg602-server: oleg602-server.virtnet: executing unload_modules_local oleg602-server: kmemleak disabled modules unloaded.