-----============= acceptance-small: conf-sanity ============----- Tue Oct 7 09:12:06 EDT 2025 oleg459-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32newtarball 110 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a skipping tests SLOW=no: 45 69 106 111 Stopping clients: oleg459-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg459-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg459-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg459-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg459-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg459-server oleg459-server: oleg459-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg459-server' oleg459-server: oleg459-server.virtnet: executing load_modules_local oleg459-server: Loading modules from /home/green/git/lustre-release/lustre oleg459-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg459-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg459-server: detected 4 online CPUs by sysfs oleg459-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg459-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg459-server: oleg459-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg459-client: oleg459-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg459-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg459-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg459-server: oleg459-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg459-client: oleg459-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg459-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg459-server: oleg459-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 40 oleg459-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg459-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg459-server stop mds service on oleg459-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg459-server SKIP: conf-sanity test_121 skipping ALWAYS excluded test 121 SKIP: conf-sanity test_122a skipping ALWAYS excluded test 122a == conf-sanity test 122b: Check OST sequence wouldn't change when IDIF 32bit overflows ========================================================== 09:13:27 (1759842807) Stopping clients: oleg459-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg459-client.virtnet /mnt/lustre2 (opts:-f) oleg459-server: oleg459-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg459-server' oleg459-server: oleg459-server.virtnet: executing load_modules_local oleg459-server: Loading modules from /home/green/git/lustre-release/lustre oleg459-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg459-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg459-server: detected 4 online CPUs by sysfs oleg459-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg459-server' oleg459-server: oleg459-server.virtnet: executing load_modules_local oleg459-server: Loading modules from /home/green/git/lustre-release/lustre oleg459-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg459-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg459-server: detected 4 online CPUs by sysfs oleg459-server: Force libcfs to create 2 CPU partitions fail_loc=0x00001e0 start mds service on oleg459-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg459-server: oleg459-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg459-client: oleg459-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg459-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg459-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg459-server: oleg459-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg459-client: oleg459-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg459-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg459-client.virtnet: -o user_xattr,flock oleg459-server@tcp:/lustre /mnt/lustre obdfilter.lustre-OST0000.precreate_batch=256 fail_loc=0 On OST0, 11872 inodes available. Want 100500. rc=0 - open/close 4730 (time 1759842856.82 total 10.00 last 472.99) - open/close 9456 (time 1759842866.82 total 20.00 last 472.50) total: 10000 open/close in 21.13 seconds: 473.29 ops/second - unlinked 0 (time 1759842869 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4580 (time 1759842893.69 total 10.00 last 457.93) - open/close 9040 (time 1759842903.69 total 20.00 last 445.97) total: 10000 open/close in 22.09 seconds: 452.75 ops/second - unlinked 0 (time 1759842907 ; total 0 ; last 0) total: 10000 unlinks in 12 seconds: 833.333313 unlinks/second - open/close 4452 (time 1759842930.96 total 10.00 last 445.19) - open/close 9035 (time 1759842940.96 total 20.00 last 458.23) total: 10000 open/close in 22.13 seconds: 451.89 ops/second - unlinked 0 (time 1759842944 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4499 (time 1759842968.93 total 10.00 last 449.90) - open/close 8992 (time 1759842978.93 total 20.00 last 449.30) total: 10000 open/close in 22.26 seconds: 449.15 ops/second - unlinked 0 (time 1759842982 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4625 (time 1759843006.97 total 10.00 last 462.45) - open/close 9292 (time 1759843016.97 total 20.00 last 466.62) total: 10000 open/close in 21.63 seconds: 462.27 ops/second - unlinked 0 (time 1759843020 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4599 (time 1759843044.50 total 10.00 last 459.86) - open/close 9228 (time 1759843054.50 total 20.00 last 462.80) total: 10000 open/close in 21.76 seconds: 459.53 ops/second - unlinked 0 (time 1759843057 ; total 0 ; last 0) total: 10000 unlinks in 12 seconds: 833.333313 unlinks/second - open/close 5159 (time 1759843080.71 total 10.00 last 515.83) - open/close 9936 (time 1759843090.71 total 20.01 last 477.47) total: 10000 open/close in 20.23 seconds: 494.21 ops/second - unlinked 0 (time 1759843092 ; total 0 ; last 0) total: 10000 unlinks in 11 seconds: 909.090881 unlinks/second - open/close 4784 (time 1759843114.42 total 10.00 last 478.35) - open/close 9138 (time 1759843124.42 total 20.00 last 435.38) total: 10000 open/close in 21.99 seconds: 454.71 ops/second - unlinked 0 (time 1759843127 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4605 (time 1759843152.68 total 10.00 last 460.45) - open/close 9234 (time 1759843162.68 total 20.00 last 462.81) total: 10000 open/close in 21.60 seconds: 462.88 ops/second - unlinked 0 (time 1759843165 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 4819 (time 1759843189.71 total 10.00 last 481.84) - open/close 9896 (time 1759843199.71 total 20.00 last 507.61) total: 10000 open/close in 20.19 seconds: 495.28 ops/second - unlinked 0 (time 1759843201 ; total 0 ; last 0) total: 10000 unlinks in 11 seconds: 909.090881 unlinks/second - open/close 4840 (time 1759843223.90 total 10.00 last 483.79) - open/close 9958 (time 1759843233.90 total 20.01 last 511.74) total: 10000 open/close in 20.09 seconds: 497.71 ops/second - unlinked 0 (time 1759843235 ; total 0 ; last 0) total: 10000 unlinks in 11 seconds: 909.090881 unlinks/second UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 383616 5760 375808 2% /mnt/lustre[MDT:0] lustre-OST0000_UUID 382976 4096 376832 2% /mnt/lustre[OST:0] filesystem_summary: 382976 4096 376832 2% /mnt/lustre 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180747 s, 5.8 MB/s fail_loc=0x80000204 Failing ost1 on oleg459-server Stopping /mnt/lustre-ost1 (opts:) on oleg459-server reboot facets: ost1 Failover ost1 to oleg459-server mount facets: ost1 Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg459-server: oleg459-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg459-client: oleg459-server: ssh exited with exit code 1 Started lustre-OST0000 oleg459-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec - open/close 0 (time 1759843276.21 total 3.66 last 0.00) total: 100 open/close in 4.25 seconds: 23.55 ops/second umount lustre on /mnt/lustre..... Stopping client oleg459-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg459-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg459-server stop mds service on oleg459-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg459-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg459-server' oleg459-server: oleg459-server.virtnet: executing unload_modules_local oleg459-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg459-server: LNET ready to unload oleg459-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded. PASS 122b (491s) SKIP: conf-sanity test_123aa skipping ALWAYS excluded test 123aa SKIP: conf-sanity test_123ab skipping ALWAYS excluded test 123ab SKIP: conf-sanity test_123ac skipping ALWAYS excluded test 123ac SKIP: conf-sanity test_123ad skipping ALWAYS excluded test 123ad SKIP: conf-sanity test_123ae skipping ALWAYS excluded test 123ae SKIP: conf-sanity test_123af skipping ALWAYS excluded test 123af SKIP: conf-sanity test_123ag skipping ALWAYS excluded test 123ag == conf-sanity test 123ai: llog_print display all non skipped records ========================================================== 09:21:43 (1759843303) start mds service on oleg459-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg459-server' oleg459-server: oleg459-server.virtnet: executing load_modules_local oleg459-server: Loading modules from /home/green/git/lustre-release/lustre oleg459-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg459-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg459-server: detected 4 online CPUs by sysfs oleg459-server: Force libcfs to create 2 CPU partitions oleg459-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg459-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg459-server: oleg459-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg459-client: oleg459-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg459-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg459-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg459-server: oleg459-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg459-client: oleg459-server: ssh exited with exit code 1 Started lustre-OST0000 oleg459-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg459-client.virtnet: -o user_xattr,flock oleg459-server@tcp:/lustre /mnt/lustre oleg459-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory pdsh@oleg459-client: oleg459-server: ssh exited with exit code 2 - { index: 394, event: set_param, device: general, parameter: timeout, value: 129 } cleanup test 123ai timeout=20 timeout=20 PASS 123ai (72s) SKIP: conf-sanity test_123F skipping ALWAYS excluded test 123F == conf-sanity test 124: check failover after replace_nids ========================================================== 09:22:55 (1759843375) SKIP: conf-sanity test_124 needs >= 2 MDTs SKIP 124 (1s) == conf-sanity test 125: check l_tunedisk only tunes OSTs and their slave devices ========================================================== 09:22:56 (1759843376) SKIP: conf-sanity test_125 ldiskfs only test SKIP 125 (1s) SKIP: conf-sanity test_126 skipping ALWAYS excluded test 126 == conf-sanity test 127: direct io overwrite on full ost ========================================================== 09:22:58 (1759843378) SKIP: conf-sanity test_127 ldiskfs only test SKIP 127 (1s) == conf-sanity test 128: Force using remote logs with --nolocallogs ========================================================== 09:22:59 (1759843379) SKIP: conf-sanity test_128 need separate mgs device SKIP 128 (1s) SKIP: conf-sanity test_129 skipping ALWAYS excluded test 129 == conf-sanity test 130: re-register an MDT after writeconf ========================================================== 09:23:01 (1759843381) SKIP: conf-sanity test_130 needs >= 2 MDTs SKIP 130 (1s) == conf-sanity test 131: MDT backup restore with project ID ========================================================== 09:23:02 (1759843382) SKIP: conf-sanity test_131 ldiskfs only test SKIP 131 (1s) SKIP: conf-sanity test_132 skipping ALWAYS excluded test 132 Stopping clients: oleg459-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg459-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg459-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg459-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg459-server oleg459-server: oleg459-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg459-server' oleg459-server: oleg459-server.virtnet: executing load_modules_local oleg459-server: Loading modules from /home/green/git/lustre-release/lustre oleg459-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg459-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg459-server: detected 4 online CPUs by sysfs oleg459-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 693 sec =========== 09:23:40 (1759843420)