-----============= acceptance-small: conf-sanity ============----- Mon Jun 30 17:31:04 EDT 2025 oleg435-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32b 32c 32newtarball 110 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg435-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg435-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg435-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg435-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg435-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg435-server oleg435-server: oleg435-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg435-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg435-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg435-server: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 40 oleg435-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg435-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg435-server stop mds service on oleg435-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg435-server == conf-sanity test 45: long unlink handling in ptlrpcd == 17:32:11 (1751319131) start mds service on oleg435-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg435-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Started lustre-OST0000 oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg435-client.virtnet: -o user_xattr,flock oleg435-server@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg435-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg435-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: '/mnt/lustre': Cannot send after transport endpoint shutdown fail_loc=0x0 start mds service on oleg435-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg435-client.virtnet: -o user_xattr,flock oleg435-server@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg435-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg435-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg435-server stop mds service on oleg435-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg435-server LNET ready to unload unloading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing unload_modules_local oleg435-server: LNET ready to unload modules unloaded. pdsh@oleg435-client: oleg435-client: ssh exited with exit code 2 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 2 pdsh@oleg435-client: oleg435-client: ssh exited with exit code 2 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 2 PASS 45 (125s) == conf-sanity test 69: replace an OST with the same index ========================================================== 17:34:16 (1751319256) start mds service on oleg435-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_size: failed to read coresize from /sys/module/intel_rapl libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg435-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg435-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg435-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Started lustre-OST0000 oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg435-client.virtnet: -o user_xattr,flock oleg435-server@tcp:/lustre /mnt/lustre On OST0, 11872 inodes available. Want 99872. rc=0 - open/close 3840 (time 1751319300.75 total 10.00 last 383.96) - open/close 7801 (time 1751319310.75 total 20.00 last 396.05) total: 10000 open/close in 24.12 seconds: 414.65 ops/second - unlinked 0 (time 1751319315 ; total 0 ; last 0) total: 10000 unlinks in 11 seconds: 909.090881 unlinks/second - open/close 5018 (time 1751319337.41 total 10.00 last 501.74) - open/close 9996 (time 1751319347.41 total 20.00 last 497.80) total: 10000 open/close in 20.01 seconds: 499.83 ops/second - unlinked 0 (time 1751319348 ; total 0 ; last 0) total: 10000 unlinks in 11 seconds: 909.090881 unlinks/second - open/close 4486 (time 1751319370.03 total 10.00 last 448.56) - open/close 8986 (time 1751319380.03 total 20.00 last 449.95) total: 10000 open/close in 22.21 seconds: 450.25 ops/second - unlinked 0 (time 1751319383 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4956 (time 1751319408.00 total 10.00 last 495.51) total: 10000 open/close in 19.17 seconds: 521.64 ops/second - unlinked 0 (time 1751319418 ; total 0 ; last 0) total: 10000 unlinks in 10 seconds: 1000.000000 unlinks/second - open/close 6005 (time 1751319438.86 total 10.00 last 600.49) total: 10000 open/close in 17.68 seconds: 565.66 ops/second - unlinked 0 (time 1751319447 ; total 0 ; last 0) total: 10000 unlinks in 11 seconds: 909.090881 unlinks/second - open/close 5047 (time 1751319469.71 total 10.00 last 504.70) - open/close 9351 (time 1751319479.71 total 20.00 last 430.38) total: 10000 open/close in 21.60 seconds: 463.07 ops/second - unlinked 0 (time 1751319482 ; total 0 ; last 0) total: 10000 unlinks in 15 seconds: 666.666687 unlinks/second - open/close 3770 (time 1751319508.25 total 10.00 last 376.98) - open/close 7510 (time 1751319518.25 total 20.00 last 373.95) total: 10000 open/close in 27.58 seconds: 362.59 ops/second - unlinked 0 (time 1751319527 ; total 1 ; last 1) total: 10000 unlinks in 15 seconds: 666.666687 unlinks/second - open/close 3376 (time 1751319552.78 total 10.00 last 337.56) - open/close 6649 (time 1751319562.78 total 20.00 last 327.27) total: 10000 open/close in 28.85 seconds: 346.66 ops/second - unlinked 0 (time 1751319572 ; total 0 ; last 0) total: 10000 unlinks in 19 seconds: 526.315796 unlinks/second - open/close 3155 (time 1751319602.24 total 10.00 last 315.40) - open/close 6074 (time 1751319612.24 total 20.01 last 291.81) - open/close 9842 (time 1751319622.24 total 30.01 last 376.75) total: 10000 open/close in 30.51 seconds: 327.81 ops/second - unlinked 0 (time 1751319623 ; total 0 ; last 0) total: 10000 unlinks in 16 seconds: 625.000000 unlinks/second - open/close 3379 (time 1751319650.78 total 10.00 last 337.80) - open/close 6457 (time 1751319660.78 total 20.01 last 307.72) - open/close 9498 (time 1751319670.78 total 30.01 last 304.05) total: 10000 open/close in 31.61 seconds: 316.34 ops/second - unlinked 0 (time 1751319673 ; total 0 ; last 0) total: 10000 unlinks in 15 seconds: 666.666687 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg435-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg435-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg435-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.204.135@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.204.135@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg435-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg435-server: oleg435-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 1 Started lustre-OST0000 oleg435-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg435-server: oleg435-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 40 oleg435-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 4 sec mount lustre on /mnt/lustre..... Starting client: oleg435-client.virtnet: -o user_xattr,flock oleg435-server@tcp:/lustre /mnt/lustre On OST0, 291 used inodes rc=0 umount lustre on /mnt/lustre..... Stopping client oleg435-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg435-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg435-server stop mds service on oleg435-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg435-server LNET ready to unload unloading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing unload_modules_local oleg435-server: LNET ready to unload modules unloaded. pdsh@oleg435-client: oleg435-client: ssh exited with exit code 2 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 2 pdsh@oleg435-client: oleg435-client: ssh exited with exit code 2 pdsh@oleg435-client: oleg435-server: ssh exited with exit code 2 PASS 69 (469s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 17:42:05 (1751319725) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (2s) Stopping clients: oleg435-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg435-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg435-client: oleg435-server: ssh exited with exit code 2 oleg435-server: oleg435-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg435-server' oleg435-server: oleg435-server.virtnet: executing load_modules_local oleg435-server: Loading modules from /home/green/git/lustre-release/lustre oleg435-server: detected 4 online CPUs by sysfs oleg435-server: Force libcfs to create 2 CPU partitions oleg435-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg435-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 683 sec =========== 17:42:27 (1751319747)