-----============= acceptance-small: conf-sanity ============----- Mon Jun 30 17:30:41 EDT 2025 oleg226-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex: No such file or directory excepting tests: 32newtarball 110 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a skipping tests SLOW=no: 45 69 106 111 Stopping clients: oleg226-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg226-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg226-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg226-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg226-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg226-server oleg226-server: oleg226-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg226-server' oleg226-server: oleg226-server.virtnet: executing load_modules_local oleg226-server: Loading modules from /home/green/git/lustre-release/lustre oleg226-server: detected 4 online CPUs by sysfs oleg226-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg226-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg226-server: oleg226-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg226-client: oleg226-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg226-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg226-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg226-server: oleg226-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg226-client: oleg226-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg226-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg226-server: oleg226-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 40 oleg226-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg226-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg226-server stop mds service on oleg226-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg226-server SKIP: conf-sanity test_121 skipping ALWAYS excluded test 121 SKIP: conf-sanity test_122a skipping ALWAYS excluded test 122a == conf-sanity test 122b: Check OST sequence wouldn't change when IDIF 32bit overflows ========================================================== 17:31:52 (1751319112) Stopping clients: oleg226-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg226-client.virtnet /mnt/lustre2 (opts:-f) oleg226-server: oleg226-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory loading modules on: 'oleg226-server' oleg226-server: oleg226-server.virtnet: executing load_modules_local oleg226-server: Loading modules from /home/green/git/lustre-release/lustre oleg226-server: detected 4 online CPUs by sysfs oleg226-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory loading modules on: 'oleg226-server' oleg226-server: oleg226-server.virtnet: executing load_modules_local oleg226-server: Loading modules from /home/green/git/lustre-release/lustre oleg226-server: detected 4 online CPUs by sysfs oleg226-server: Force libcfs to create 2 CPU partitions oleg226-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory fail_loc=0x00001e0 start mds service on oleg226-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg226-server: oleg226-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg226-client: oleg226-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg226-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg226-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg226-server: oleg226-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg226-client: oleg226-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg226-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg226-client.virtnet: -o user_xattr,flock oleg226-server@tcp:/lustre /mnt/lustre obdfilter.lustre-OST0000.precreate_batch=256 fail_loc=0 On OST0, 11872 inodes available. Want 100500. rc=0 - open/close 5049 (time 1751319150.95 total 10.00 last 504.81) total: 10000 open/close in 19.43 seconds: 514.59 ops/second - unlinked 0 (time 1751319161 ; total 0 ; last 0) total: 10000 unlinks in 11 seconds: 909.090881 unlinks/second - open/close 6388 (time 1751319183.10 total 10.00 last 638.72) total: 10000 open/close in 16.03 seconds: 623.74 ops/second - unlinked 0 (time 1751319189 ; total 0 ; last 0) total: 10000 unlinks in 9 seconds: 1111.111084 unlinks/second - open/close 6144 (time 1751319210.01 total 10.00 last 614.18) total: 10000 open/close in 16.35 seconds: 611.60 ops/second - unlinked 0 (time 1751319217 ; total 0 ; last 0) total: 10000 unlinks in 9 seconds: 1111.111084 unlinks/second - open/close 6820 (time 1751319236.95 total 10.00 last 681.95) total: 10000 open/close in 14.89 seconds: 671.38 ops/second - unlinked 0 (time 1751319242 ; total 0 ; last 0) total: 10000 unlinks in 9 seconds: 1111.111084 unlinks/second - open/close 6369 (time 1751319262.50 total 10.00 last 636.83) total: 10000 open/close in 15.56 seconds: 642.74 ops/second - unlinked 0 (time 1751319268 ; total 0 ; last 0) total: 10000 unlinks in 9 seconds: 1111.111084 unlinks/second - open/close 6777 (time 1751319288.65 total 10.00 last 677.69) total: 10000 open/close in 14.86 seconds: 672.93 ops/second - unlinked 0 (time 1751319294 ; total 0 ; last 0) total: 10000 unlinks in 9 seconds: 1111.111084 unlinks/second - open/close 6387 (time 1751319314.07 total 10.00 last 638.65) total: 10000 open/close in 15.48 seconds: 645.95 ops/second - unlinked 0 (time 1751319320 ; total 0 ; last 0) total: 10000 unlinks in 8 seconds: 1250.000000 unlinks/second - open/close 7014 (time 1751319339.82 total 10.00 last 701.32) total: 10000 open/close in 15.19 seconds: 658.42 ops/second - unlinked 0 (time 1751319345 ; total 0 ; last 0) total: 10000 unlinks in 11 seconds: 909.090881 unlinks/second - open/close 5335 (time 1751319367.51 total 10.00 last 533.46) total: 10000 open/close in 18.32 seconds: 545.79 ops/second - unlinked 0 (time 1751319376 ; total 0 ; last 0) total: 10000 unlinks in 11 seconds: 909.090881 unlinks/second - open/close 5384 (time 1751319398.58 total 10.00 last 538.30) total: 10000 open/close in 18.34 seconds: 545.25 ops/second - unlinked 0 (time 1751319407 ; total 0 ; last 0) total: 10000 unlinks in 11 seconds: 909.090881 unlinks/second - open/close 5670 (time 1751319429.30 total 10.00 last 566.92) total: 10000 open/close in 17.83 seconds: 560.95 ops/second - unlinked 0 (time 1751319437 ; total 0 ; last 0) total: 10000 unlinks in 11 seconds: 909.090881 unlinks/second UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 383616 5760 375808 2% /mnt/lustre[MDT:0] lustre-OST0000_UUID 382976 4096 376832 2% /mnt/lustre[OST:0] filesystem_summary: 382976 4096 376832 2% /mnt/lustre 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0887401 s, 11.8 MB/s fail_loc=0x80000204 Failing ost1 on oleg226-server Stopping /mnt/lustre-ost1 (opts:) on oleg226-server reboot facets: ost1 Failover ost1 to oleg226-server mount facets: ost1 Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg226-server: oleg226-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg226-client: oleg226-server: ssh exited with exit code 1 Started lustre-OST0000 oleg226-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec - open/close 0 (time 1751319476.01 total 5.09 last 0.00) total: 100 open/close in 5.57 seconds: 17.97 ops/second umount lustre on /mnt/lustre..... Stopping client oleg226-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg226-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg226-server stop mds service on oleg226-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg226-server LNET ready to unload unloading modules on: 'oleg226-server' oleg226-server: oleg226-server.virtnet: executing unload_modules_local oleg226-server: LNET ready to unload modules unloaded. PASS 122b (383s) SKIP: conf-sanity test_123aa skipping ALWAYS excluded test 123aa SKIP: conf-sanity test_123ab skipping ALWAYS excluded test 123ab SKIP: conf-sanity test_123ac skipping ALWAYS excluded test 123ac SKIP: conf-sanity test_123ad skipping ALWAYS excluded test 123ad SKIP: conf-sanity test_123ae skipping ALWAYS excluded test 123ae SKIP: conf-sanity test_123af skipping ALWAYS excluded test 123af SKIP: conf-sanity test_123ag skipping ALWAYS excluded test 123ag == conf-sanity test 123ai: llog_print display all non skipped records ========================================================== 17:38:19 (1751319499) start mds service on oleg226-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg226-server' oleg226-server: oleg226-server.virtnet: executing load_modules_local oleg226-server: Loading modules from /home/green/git/lustre-release/lustre oleg226-server: detected 4 online CPUs by sysfs oleg226-server: Force libcfs to create 2 CPU partitions oleg226-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg226-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg226-server: oleg226-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg226-client: oleg226-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg226-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg226-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg226-server: oleg226-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg226-client: oleg226-server: ssh exited with exit code 1 Started lustre-OST0000 oleg226-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg226-client.virtnet: -o user_xattr,flock oleg226-server@tcp:/lustre /mnt/lustre oleg226-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory pdsh@oleg226-client: oleg226-server: ssh exited with exit code 2 - { index: 394, event: set_param, device: general, parameter: timeout, value: 129 } cleanup test 123ai timeout=20 timeout=20 PASS 123ai (51s) SKIP: conf-sanity test_123F skipping ALWAYS excluded test 123F == conf-sanity test 124: check failover after replace_nids ========================================================== 17:39:10 (1751319550) SKIP: conf-sanity test_124 needs >= 2 MDTs SKIP 124 (1s) == conf-sanity test 125: check l_tunedisk only tunes OSTs and their slave devices ========================================================== 17:39:11 (1751319551) SKIP: conf-sanity test_125 ldiskfs only test SKIP 125 (1s) SKIP: conf-sanity test_126 skipping ALWAYS excluded test 126 == conf-sanity test 127: direct io overwrite on full ost ========================================================== 17:39:12 (1751319552) SKIP: conf-sanity test_127 ldiskfs only test SKIP 127 (1s) == conf-sanity test 128: Force using remote logs with --nolocallogs ========================================================== 17:39:13 (1751319553) SKIP: conf-sanity test_128 need separate mgs device SKIP 128 (1s) SKIP: conf-sanity test_129 skipping ALWAYS excluded test 129 == conf-sanity test 130: re-register an MDT after writeconf ========================================================== 17:39:14 (1751319554) SKIP: conf-sanity test_130 needs >= 2 MDTs SKIP 130 (1s) == conf-sanity test 131: MDT backup restore with project ID ========================================================== 17:39:15 (1751319555) SKIP: conf-sanity test_131 ldiskfs only test SKIP 131 (0s) SKIP: conf-sanity test_132 skipping ALWAYS excluded test 132 Stopping clients: oleg226-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg226-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg226-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg226-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg226-server oleg226-server: oleg226-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg226-server' oleg226-server: oleg226-server.virtnet: executing load_modules_local oleg226-server: Loading modules from /home/green/git/lustre-release/lustre oleg226-server: detected 4 online CPUs by sysfs oleg226-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 543 sec =========== 17:39:45 (1751319585)