-----============= acceptance-small: conf-sanity ============----- Fri Jun 12 12:14:39 EDT 2026 oleg627-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32newtarball 110 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a skipping tests SLOW=no: 45 69 106 111 Stopping clients: oleg627-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg627-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg627-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg627-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg627-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg627-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg627-server oleg627-server: oleg627-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg627-server' oleg627-server: oleg627-server.virtnet: executing load_modules_local oleg627-server: Loading modules from /home/green/git/lustre-release/lustre oleg627-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg627-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg627-server: detected 4 online CPUs by sysfs oleg627-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format mds2: lustre-mdt2/mdt2 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg627-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 start mds service on oleg627-server Starting mds2: -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Commit the device label on lustre-mdt2/mdt2 Started lustre-MDT0001 oleg627-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg627-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg627-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg627-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg627-server: oleg627-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 40 oleg627-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg627-server: oleg627-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 40 oleg627-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg627-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg627-server stop mds service on oleg627-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg627-server stop mds service on oleg627-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg627-server == conf-sanity test 154: expand .. on rename after MDT backup restore ========================================================== 12:18:30 (1781281110) SKIP: conf-sanity test_154 ldiskfs only test SKIP 154 (3s) SKIP: conf-sanity test_121 skipping ALWAYS excluded test 121 SKIP: conf-sanity test_122a skipping ALWAYS excluded test 122a == conf-sanity test 122b: Check OST sequence wouldn't change when IDIF 32bit overflows ========================================================== 12:18:36 (1781281116) Stopping clients: oleg627-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg627-client.virtnet /mnt/lustre2 (opts:-f) oleg627-server: oleg627-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg627-server' oleg627-server: oleg627-server.virtnet: executing load_modules_local oleg627-server: Loading modules from /home/green/git/lustre-release/lustre oleg627-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg627-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg627-server: detected 4 online CPUs by sysfs oleg627-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format mds2: lustre-mdt2/mdt2 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg627-server' oleg627-server: oleg627-server.virtnet: executing load_modules_local oleg627-server: Loading modules from /home/green/git/lustre-release/lustre oleg627-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg627-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg627-server: detected 4 online CPUs by sysfs oleg627-server: Force libcfs to create 2 CPU partitions fail_loc=0x00001e0 start mds service on oleg627-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 start mds service on oleg627-server Starting mds2: -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Commit the device label on lustre-mdt2/mdt2 Started lustre-MDT0001 oleg627-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg627-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg627-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg627-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg627-client.virtnet: -o user_xattr,flock oleg627-server@tcp:/lustre /mnt/lustre obdfilter.lustre-OST0000.precreate_batch=256 fail_loc=0 On OST0, 11872 inodes available. Want 100500. rc=0 - open/close 793 (time 1781281269.81 total 10.01 last 79.26) - open/close 1611 (time 1781281279.81 total 20.01 last 81.74) - open/close 2418 (time 1781281289.82 total 30.02 last 80.67) - open/close 3258 (time 1781281299.82 total 40.02 last 83.99) - open/close 4073 (time 1781281309.83 total 50.03 last 81.38) - open/close 4889 (time 1781281319.85 total 60.04 last 81.50) - open/close 5677 (time 1781281329.85 total 70.05 last 78.78) - open/close 6488 (time 1781281339.86 total 80.06 last 81.00) - open/close 7271 (time 1781281349.87 total 90.07 last 78.23) - open/close 8090 (time 1781281359.87 total 100.07 last 81.88) - open/close 8879 (time 1781281369.88 total 110.08 last 78.86) - open/close 9692 (time 1781281379.88 total 120.08 last 81.28) total: 10000 open/close in 123.63 seconds: 80.89 ops/second - unlinked 0 (time 1781281387 ; total 0 ; last 0) total: 10000 unlinks in 72 seconds: 138.888885 unlinks/second - open/close 790 (time 1781281474.12 total 10.01 last 78.93) - open/close 1597 (time 1781281484.13 total 20.02 last 80.64) - open/close 2438 (time 1781281494.13 total 30.02 last 84.09) - open/close 3265 (time 1781281504.15 total 40.04 last 82.51) - open/close 4105 (time 1781281514.17 total 50.05 last 83.90) - open/close 4915 (time 1781281524.17 total 60.05 last 80.99) - open/close 5794 (time 1781281534.17 total 70.06 last 87.85) - open/close 6692 (time 1781281544.17 total 80.06 last 89.80) - open/close 7542 (time 1781281554.18 total 90.06 last 84.97) - open/close 8426 (time 1781281564.18 total 100.07 last 88.36) - open/close 9345 (time 1781281574.19 total 110.07 last 91.85) total: 10000 open/close in 117.29 seconds: 85.26 ops/second - unlinked 0 (time 1781281584 ; total 0 ; last 0) total: 10000 unlinks in 70 seconds: 142.857147 unlinks/second - open/close 821 (time 1781281668.65 total 10.01 last 82.03) - open/close 1649 (time 1781281678.66 total 20.02 last 82.68) - open/close 2463 (time 1781281688.67 total 30.03 last 81.38) - open/close 3333 (time 1781281698.68 total 40.04 last 86.87) - open/close 4186 (time 1781281708.69 total 50.05 last 85.23) - open/close 5019 (time 1781281718.70 total 60.05 last 83.26) - open/close 5925 (time 1781281728.70 total 70.06 last 90.53) - open/close 6776 (time 1781281738.71 total 80.07 last 85.02) - open/close 7620 (time 1781281748.72 total 90.08 last 84.36) - open/close 8517 (time 1781281758.72 total 100.08 last 89.67) - open/close 9369 (time 1781281768.73 total 110.09 last 85.13) total: 10000 open/close in 117.97 seconds: 84.76 ops/second - unlinked 0 (time 1781281780 ; total 0 ; last 0) total: 10000 unlinks in 65 seconds: 153.846161 unlinks/second - open/close 833 (time 1781281858.77 total 10.00 last 83.29) - open/close 1753 (time 1781281868.79 total 20.01 last 91.88) - open/close 2636 (time 1781281878.79 total 30.02 last 88.28) - open/close 3537 (time 1781281888.79 total 40.02 last 90.05) - open/close 4482 (time 1781281898.81 total 50.04 last 94.34) - open/close 5527 (time 1781281908.82 total 60.04 last 104.44) - open/close 6612 (time 1781281918.82 total 70.05 last 108.46) - open/close 7530 (time 1781281928.84 total 80.07 last 91.64) - open/close 8502 (time 1781281938.84 total 90.07 last 97.16) - open/close 9399 (time 1781281948.86 total 100.08 last 89.58) total: 10000 open/close in 106.86 seconds: 93.58 ops/second - unlinked 0 (time 1781281958 ; total 0 ; last 0) total: 10000 unlinks in 60 seconds: 166.666672 unlinks/second - open/close 936 (time 1781282031.93 total 10.01 last 93.53) - open/close 1842 (time 1781282041.94 total 20.02 last 90.50) - open/close 2880 (time 1781282051.95 total 30.03 last 103.71) - open/close 3780 (time 1781282061.96 total 40.04 last 89.90) - open/close 4784 (time 1781282071.96 total 50.04 last 100.40) - open/close 5706 (time 1781282081.97 total 60.04 last 92.19) - open/close 6625 (time 1781282091.97 total 70.04 last 91.85) - open/close 7573 (time 1781282101.97 total 80.05 last 94.77) - open/close 8637 (time 1781282111.98 total 90.05 last 106.35) - open/close 9711 (time 1781282121.98 total 100.05 last 107.38) total: 10000 open/close in 102.95 seconds: 97.14 ops/second - unlinked 0 (time 1781282128 ; total 0 ; last 0) total: 10000 unlinks in 64 seconds: 156.250000 unlinks/second - open/close 867 (time 1781282206.93 total 10.02 last 86.56) - open/close 1864 (time 1781282216.94 total 20.02 last 99.62) - open/close 2783 (time 1781282226.94 total 30.03 last 91.88) - open/close 3729 (time 1781282236.95 total 40.03 last 94.53) - open/close 4620 (time 1781282246.95 total 50.04 last 89.08) - open/close 5561 (time 1781282256.97 total 60.05 last 93.96) - open/close 6511 (time 1781282266.97 total 70.06 last 94.95) - open/close 7374 (time 1781282276.98 total 80.07 last 86.22) - open/close 8365 (time 1781282286.98 total 90.07 last 99.09) - open/close 9229 (time 1781282296.99 total 100.07 last 86.34) total: 10000 open/close in 109.03 seconds: 91.72 ops/second - unlinked 0 (time 1781282309 ; total 0 ; last 0) total: 10000 unlinks in 67 seconds: 149.253738 unlinks/second - open/close 975 (time 1781282389.75 total 10.00 last 97.46) - open/close 1922 (time 1781282399.75 total 20.01 last 94.67) - open/close 2898 (time 1781282409.75 total 30.01 last 97.56) - open/close 3850 (time 1781282419.75 total 40.01 last 95.19) - open/close 4761 (time 1781282429.76 total 50.02 last 91.03) - open/close 5637 (time 1781282439.76 total 60.02 last 87.60) - open/close 6550 (time 1781282449.77 total 70.02 last 91.26) - open/close 7420 (time 1781282459.77 total 80.03 last 86.92) - open/close 8329 (time 1781282469.78 total 90.04 last 90.87) - open/close 9191 (time 1781282479.78 total 100.04 last 86.16) total: 10000 open/close in 109.58 seconds: 91.26 ops/second - unlinked 0 (time 1781282493 ; total 0 ; last 0) total: 10000 unlinks in 68 seconds: 147.058823 unlinks/second - open/close 814 (time 1781282575.95 total 10.01 last 81.33) - open/close 1662 (time 1781282585.96 total 20.02 last 84.70) - open/close 2487 (time 1781282595.96 total 30.02 last 82.50) - open/close 3353 (time 1781282605.97 total 40.03 last 86.51) - open/close 4306 (time 1781282615.97 total 50.04 last 95.26) - open/close 5257 (time 1781282625.98 total 60.04 last 95.05) - open/close 6152 (time 1781282635.98 total 70.05 last 89.46) - open/close 7013 (time 1781282645.99 total 80.05 last 86.08) - open/close 7820 (time 1781282655.99 total 90.05 last 80.67) - open/close 8660 (time 1781282666.00 total 100.06 last 83.94) - open/close 9517 (time 1781282676.00 total 110.06 last 85.64) total: 10000 open/close in 115.76 seconds: 86.39 ops/second - unlinked 0 (time 1781282685 ; total 0 ; last 0) total: 10000 unlinks in 70 seconds: 142.857147 unlinks/second - open/close 823 (time 1781282769.36 total 10.00 last 82.29) - open/close 1766 (time 1781282779.36 total 20.00 last 94.29) - open/close 2673 (time 1781282789.37 total 30.01 last 90.62) - open/close 3636 (time 1781282799.38 total 40.02 last 96.26) - open/close 4541 (time 1781282809.38 total 50.02 last 90.47) - open/close 5407 (time 1781282819.38 total 60.02 last 86.58) - open/close 6233 (time 1781282829.39 total 70.03 last 82.49) - open/close 7032 (time 1781282839.40 total 80.04 last 79.83) - open/close 7823 (time 1781282849.41 total 90.05 last 79.04) - open/close 8682 (time 1781282859.41 total 100.05 last 85.89) - open/close 9567 (time 1781282869.42 total 110.06 last 88.46) total: 10000 open/close in 115.34 seconds: 86.70 ops/second - unlinked 0 (time 1781282878 ; total 0 ; last 0) total: 10000 unlinks in 70 seconds: 142.857147 unlinks/second - open/close 899 (time 1781282961.95 total 10.01 last 89.83) - open/close 1849 (time 1781282971.96 total 20.01 last 94.94) - open/close 2755 (time 1781282981.97 total 30.02 last 90.50) - open/close 3687 (time 1781282991.97 total 40.03 last 93.18) - open/close 4652 (time 1781283001.97 total 50.03 last 96.45) - open/close 5508 (time 1781283011.98 total 60.03 last 85.59) - open/close 6415 (time 1781283021.98 total 70.03 last 90.68) - open/close 7283 (time 1781283031.98 total 80.04 last 86.78) - open/close 8139 (time 1781283041.99 total 90.04 last 85.54) - open/close 9008 (time 1781283051.99 total 100.05 last 86.89) - open/close 9867 (time 1781283061.99 total 110.05 last 85.89) total: 10000 open/close in 111.65 seconds: 89.56 ops/second - unlinked 0 (time 1781283067 ; total 0 ; last 0) total: 10000 unlinks in 61 seconds: 163.934433 unlinks/second - open/close 889 (time 1781283142.16 total 10.02 last 88.72) - open/close 1973 (time 1781283152.17 total 20.03 last 108.32) - open/close 2941 (time 1781283162.17 total 30.03 last 96.79) - open/close 3846 (time 1781283172.18 total 40.04 last 90.43) - open/close 4667 (time 1781283182.18 total 50.04 last 82.10) - open/close 5601 (time 1781283192.18 total 60.04 last 93.36) - open/close 6372 (time 1781283202.19 total 70.04 last 77.07) - open/close 7167 (time 1781283212.19 total 80.05 last 79.47) - open/close 8017 (time 1781283222.20 total 90.06 last 84.93) - open/close 8876 (time 1781283232.20 total 100.06 last 85.85) - open/close 9714 (time 1781283242.22 total 110.08 last 83.66) total: 10000 open/close in 113.64 seconds: 88.00 ops/second - unlinked 0 (time 1781283249 ; total 0 ; last 0) total: 10000 unlinks in 65 seconds: 153.846161 unlinks/second UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 383616 5376 376192 2% /mnt/lustre[MDT:0] lustre-MDT0001_UUID 383616 3072 378496 1% /mnt/lustre[MDT:1] lustre-OST0000_UUID 382976 4096 376832 2% /mnt/lustre[OST:0] filesystem_summary: 382976 4096 376832 2% /mnt/lustre 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.371196 s, 2.8 MB/s fail_loc=0x80000204 Failing ost1 on oleg627-server Stopping /mnt/lustre-ost1 (opts:) on oleg627-server reboot facets: ost1 Failover ost1 to oleg627-server mount facets: ost1 Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Started lustre-OST0000 oleg627-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec total: 100 open/close in 2.16 seconds: 46.27 ops/second umount lustre on /mnt/lustre..... Stopping client oleg627-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg627-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg627-server stop mds service on oleg627-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg627-server stop mds service on oleg627-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg627-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg627-server' oleg627-server: oleg627-server.virtnet: executing unload_modules_local oleg627-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg627-server: LNET ready to unload oleg627-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded. PASS 122b (2283s) SKIP: conf-sanity test_123aa skipping ALWAYS excluded test 123aa SKIP: conf-sanity test_123ab skipping ALWAYS excluded test 123ab SKIP: conf-sanity test_123ac skipping ALWAYS excluded test 123ac SKIP: conf-sanity test_123ad skipping ALWAYS excluded test 123ad SKIP: conf-sanity test_123ae skipping ALWAYS excluded test 123ae SKIP: conf-sanity test_123af skipping ALWAYS excluded test 123af SKIP: conf-sanity test_123ag skipping ALWAYS excluded test 123ag == conf-sanity test 123ai: llog_print display all non skipped records ========================================================== 12:56:49 (1781283409) start mds service on oleg627-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg627-server' oleg627-server: oleg627-server.virtnet: executing load_modules_local oleg627-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg627-server: Loading modules from /home/green/git/lustre-release/lustre oleg627-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg627-server: detected 4 online CPUs by sysfs oleg627-server: Force libcfs to create 2 CPU partitions oleg627-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg627-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg627-server Starting mds2: -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg627-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg627-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg627-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Started lustre-OST0000 oleg627-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg627-client.virtnet: -o user_xattr,flock oleg627-server@tcp:/lustre /mnt/lustre oleg627-server: params: OBD_IOC_LLOG_PRINT failed: No such file or directory pdsh@oleg627-client: oleg627-server: ssh exited with exit code 2 - { index: 394, event: set_param, device: general, parameter: timeout, value: 129 } cleanup test 123ai timeout=20 timeout=20 PASS 123ai (188s) SKIP: conf-sanity test_123F skipping ALWAYS excluded test 123F == conf-sanity test 124: check failover after replace_nids ========================================================== 12:59:59 (1781283599) SKIP: conf-sanity test_124 needs MDT failover setup SKIP 124 (3s) == conf-sanity test 125: check l_tunedisk only tunes OSTs and their slave devices ========================================================== 13:00:02 (1781283602) SKIP: conf-sanity test_125 ldiskfs only test SKIP 125 (3s) SKIP: conf-sanity test_126 skipping ALWAYS excluded test 126 == conf-sanity test 127: direct io overwrite on full ost ========================================================== 13:00:07 (1781283607) SKIP: conf-sanity test_127 ldiskfs only test SKIP 127 (3s) == conf-sanity test 128: Force using remote logs with --nolocallogs ========================================================== 13:00:10 (1781283610) SKIP: conf-sanity test_128 need separate mgs device SKIP 128 (3s) SKIP: conf-sanity test_129 skipping ALWAYS excluded test 129 == conf-sanity test 130: re-register an MDT after writeconf ========================================================== 13:00:14 (1781283614) Checking servers environments Checking clients oleg627-client.virtnet environments /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg627-server' oleg627-server: oleg627-server.virtnet: executing load_modules_local oleg627-server: Loading modules from /home/green/git/lustre-release/lustre oleg627-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg627-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg627-server: detected 4 online CPUs by sysfs oleg627-server: Force libcfs to create 2 CPU partitions Setup mgs, mdt, osts Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg627-server: mount.lustre: according to /etc/mtab lustre-mdt1/mdt1 is already mounted on /mnt/lustre-mds1 pdsh@oleg627-client: oleg627-server: ssh exited with exit code 17 Start of lustre-mdt1/mdt1 on mds1 failed 17 Starting mds2: -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg627-server: mount.lustre: according to /etc/mtab lustre-mdt2/mdt2 is already mounted on /mnt/lustre-mds2 pdsh@oleg627-client: oleg627-server: ssh exited with exit code 17 Start of lustre-mdt2/mdt2 on mds2 failed 17 Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg627-server: mount.lustre: according to /etc/mtab lustre-ost1/ost1 is already mounted on /mnt/lustre-ost1 pdsh@oleg627-client: oleg627-server: ssh exited with exit code 17 Start of lustre-ost1/ost1 on ost1 failed 17 Starting ost2: -o localrecov lustre-ost2/ost2 /mnt/lustre-ost2 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Commit the device label on lustre-ost2/ost2 Started lustre-OST0001 mount lustre on /mnt/lustre..... Starting client: oleg627-client.virtnet: -o user_xattr,flock oleg627-server@tcp:/lustre /mnt/lustre mount.lustre: according to /etc/mtab oleg627-server@tcp:/lustre is already mounted on /mnt/lustre Starting client oleg627-client.virtnet: -o user_xattr,flock oleg627-server@tcp:/lustre /mnt/lustre Started clients oleg627-client.virtnet: 192.168.206.127@tcp:/lustre on /mnt/lustre type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt) Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff91928519f000.idle_timeout=debug osc.lustre-OST0001-osc-ffff91928519f000.idle_timeout=debug setting jobstats to procname_uid Setting lustre.sys.jobid_var from disable to procname_uid Waiting 90s for 'procname_uid' Updated after 6s: want 'procname_uid' got 'procname_uid' disable quota as required stop mds service on oleg627-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg627-server oleg627-server: oleg627-server: tunefs.lustre FATAL: Device lustre-mdt2/mdt2 has not been formatted with mkfs.lustre oleg627-server: tunefs.lustre: exiting with 19 (No such device) pdsh@oleg627-client: oleg627-server: ssh exited with exit code 19 checking for existing Lustre data: not found start mds service on oleg627-server Starting mds2: -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg627-server: oleg627-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg627-client: oleg627-server: ssh exited with exit code 1 Started lustre-MDT0001 PASS 130 (78s) == conf-sanity test 131: MDT backup restore with project ID ========================================================== 13:01:32 (1781283692) SKIP: conf-sanity test_131 ldiskfs only test SKIP 131 (3s) SKIP: conf-sanity test_132 skipping ALWAYS excluded test 132 Stopping clients: oleg627-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg627-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg627-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg627-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg627-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg627-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg627-server oleg627-server: oleg627-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg627-server' oleg627-server: oleg627-server.virtnet: executing load_modules_local oleg627-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg627-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg627-server: Loading modules from /home/green/git/lustre-release/lustre oleg627-server: detected 4 online CPUs by sysfs oleg627-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format mds2: lustre-mdt2/mdt2 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 2902 sec ========== 13:03:04 (1781283784)