-----============= acceptance-small: conf-sanity ============----- Fri Apr 5 18:46:40 EDT 2024 excepting tests: 32newtarball 110 skipping tests SLOW=no: 45 69 106 111 114 Stopping clients: oleg320-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg320-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg320-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg320-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg320-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg320-server oleg320-server: oleg320-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory loading modules on: 'oleg320-server' oleg320-server: oleg320-server.virtnet: executing load_modules_local oleg320-server: Loading modules from /home/green/git/lustre-release/lustre oleg320-server: detected 4 online CPUs by sysfs oleg320-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg320-server Starting mds1: -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg320-server: oleg320-server.virtnet: executing set_default_debug -1 all pdsh@oleg320-client: oleg320-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 oleg320-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg320-server Starting ost1: -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg320-server: oleg320-server.virtnet: executing set_default_debug -1 all pdsh@oleg320-client: oleg320-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg320-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg320-server: oleg320-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg320-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg320-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg320-server stop mds service on oleg320-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg320-server debug_raw_pointers=Y debug_raw_pointers=Y == conf-sanity test 46b: online OST and MDT addition ===== 18:47:54 (1712357274) Testing with 2 OSTs and 1 MDT(s) Stopping clients: oleg320-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg320-client.virtnet /mnt/lustre2 (opts:-f) oleg320-server: oleg320-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg320-server' oleg320-server: oleg320-server.virtnet: executing load_modules_local oleg320-server: Loading modules from /home/green/git/lustre-release/lustre oleg320-server: detected 4 online CPUs by sysfs oleg320-server: Force libcfs to create 2 CPU partitions oleg320-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg320-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey start mds service on oleg320-server Starting mds1: -o localrecov /dev/mapper/mds1_flakey /mnt/lustre-mds1 oleg320-server: oleg320-server.virtnet: executing set_default_debug -1 all pdsh@oleg320-client: oleg320-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/mds1_flakey Started lustre-MDT0000 oleg320-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg320-server Starting ost1: -o localrecov /dev/mapper/ost1_flakey /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg320-server: oleg320-server.virtnet: executing set_default_debug -1 all pdsh@oleg320-client: oleg320-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost1_flakey Started lustre-OST0000 oleg320-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg320-server: oleg320-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg320-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg320-client.virtnet: -o user_xattr,flock oleg320-server@tcp:/lustre /mnt/lustre lmv.lustre-clilmv-ffff8800aa323000.qos_maxage=10 max copy 126824 KiB, should be 126828 KiB available: /usr/lib/locale /usr/lib/python3.6 /usr/lib/grub /usr/lib/rpm /usr/lib/libptlctl.a /usr/lib/sendmail.postfix /usr/lib/sendmail /usr/lib/cpp fail_val=1 debug=all debug_mb=150 debug=all debug_mb=150 rsync /usr/lib/python3.6/site-packages->/mnt/lustre/d46b.conf-sanity/python3.6/site-packages: lustre-MDT0000 Starting ost2: -o localrecov /dev/mapper/ost2_flakey /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg320-server: oleg320-server.virtnet: executing set_default_debug -1 all pdsh@oleg320-client: oleg320-server: ssh exited with exit code 1 Commit the device label on /dev/mapper/ost2_flakey Started lustre-OST0001 OST2 addition finished mount lustre on /mnt/lustre2..... Starting client: oleg320-client.virtnet: -o user_xattr,flock oleg320-server@tcp:/lustre /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 95248 1904 84688 3% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 142216 1388 126828 2% /mnt/lustre2[OST:0] lustre-OST0001_UUID 142216 1388 126828 2% /mnt/lustre2[OST:1] filesystem_summary: 284432 2776 253656 2% /mnt/lustre2 lfs mkdir: dirstripe error on '/mnt/lustre2/d46b.conf-sanity.2': No such device lfs setdirstripe: cannot create dir '/mnt/lustre2/d46b.conf-sanity.2': No such device conf-sanity test_46b: @@@@@@ FAIL: mkdir -i 1 -c -1 /mnt/lustre2/d46b.conf-sanity.2 failed Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:6974:error() = /home/green/git/lustre-release/lustre/tests/conf-sanity.sh:4225:test_46b() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7314:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7374:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7185:run_test() = /home/green/git/lustre-release/lustre/tests/conf-sanity.sh:4290:main() Dumping lctl log to /tmp/testlogs//conf-sanity.test_46b.*.1712357323.log debug=trace inode super iotrace malloc cache info ioctl neterror net warning buffs other dentry nettrace page dlmtrace error emerg ha rpctrace vfstrace reada mmap config console quota sec lfsck hsm snapshot layout debug=trace inode super iotrace malloc cache info ioctl neterror net warning buffs other dentry nettrace page dlmtrace error emerg ha rpctrace vfstrace reada mmap config console quota sec lfsck hsm snapshot layout /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4661: 13397 Killed rsync -az -e ssh $node:${prefix}.'*'.${suffix} $LOGDIR debug_mb=21 debug_mb=21 umount lustre on /mnt/lustre2..... Stopping client oleg320-client.virtnet /mnt/lustre2 (opts:) umount lustre on /mnt/lustre..... Stopping client oleg320-client.virtnet /mnt/lustre (opts:) Stopping /mnt/lustre-ost2 (opts:-f) on oleg320-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg320-server FAIL 46b (71s) debug_raw_pointers=0 debug_raw_pointers=0 Stopping clients: oleg320-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg320-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg320-server oleg320-server: oleg320-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg320-server' oleg320-server: oleg320-server.virtnet: executing load_modules_local oleg320-server: Loading modules from /home/green/git/lustre-release/lustre oleg320-server: detected 4 online CPUs by sysfs oleg320-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: /dev/mapper/mds1_flakey Format ost1: /dev/mapper/ost1_flakey Format ost2: /dev/mapper/ost2_flakey == conf-sanity test complete, duration 165 sec =========== 18:49:25 (1712357365) conf-sanity: FAIL: test_46b mkdir -i 1 -c -1 /mnt/lustre2/d46b.conf-sanity.2 failed === conf-sanity: start cleanup 18:49:26 (1712357366) === === conf-sanity: finish cleanup 18:49:26 (1712357366) ===