-----============= acceptance-small: conf-sanity ============----- Sun Nov 16 21:13:15 EST 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg152-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32b 32c 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg152-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg152-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg152-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg152-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg152-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg152-server oleg152-server: oleg152-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg152-server' oleg152-server: oleg152-server.virtnet: executing load_modules_local oleg152-server: Loading modules from /home/green/git/lustre-release/lustre oleg152-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg152-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg152-server: detected 4 online CPUs by sysfs oleg152-server: MODOPTS_LIBCFS= oleg152-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg152-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg152-server: oleg152-server.virtnet: executing set_default_debug -1 all pdsh@oleg152-client: oleg152-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg152-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg152-server: oleg152-server.virtnet: executing set_default_debug -1 all pdsh@oleg152-client: oleg152-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg152-server: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg152-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg152-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg152-server stop mds service on oleg152-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg152-server == conf-sanity test 45: long unlink handling in ptlrpcd == 21:15:33 (1763345733) start mds service on oleg152-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg152-server: oleg152-server.virtnet: executing set_default_debug -1 all pdsh@oleg152-client: oleg152-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg152-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg152-server: oleg152-server.virtnet: executing set_default_debug -1 all pdsh@oleg152-client: oleg152-server: ssh exited with exit code 1 Started lustre-OST0000 oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg152-client.virtnet: -o user_xattr,flock 192.168.201.152@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg152-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg152-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg152-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg152-server: oleg152-server.virtnet: executing set_default_debug -1 all pdsh@oleg152-client: oleg152-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg152-client.virtnet: -o user_xattr,flock 192.168.201.152@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg152-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg152-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg152-server stop mds service on oleg152-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg152-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg152-server' oleg152-server: oleg152-server.virtnet: executing unload_modules_local oleg152-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg152-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg152-client: oleg152-client: ssh exited with exit code 2 pdsh@oleg152-client: oleg152-server: ssh exited with exit code 2 pdsh@oleg152-client: oleg152-client: ssh exited with exit code 2 pdsh@oleg152-client: oleg152-server: ssh exited with exit code 2 PASS 45 (202s) == conf-sanity test 69: replace an OST with the same index ========================================================== 21:18:56 (1763345936) start mds service on oleg152-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg152-server' oleg152-server: oleg152-server.virtnet: executing load_modules_local oleg152-server: Loading modules from /home/green/git/lustre-release/lustre oleg152-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg152-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg152-server: detected 4 online CPUs by sysfs oleg152-server: MODOPTS_LIBCFS= oleg152-server: Force libcfs to create 2 CPU partitions oleg152-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg152-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg152-server: oleg152-server.virtnet: executing set_default_debug -1 all pdsh@oleg152-client: oleg152-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg152-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg152-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg152-server: oleg152-server.virtnet: executing set_default_debug -1 all pdsh@oleg152-client: oleg152-server: ssh exited with exit code 1 Started lustre-OST0000 oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg152-client.virtnet: -o user_xattr,flock 192.168.201.152@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 11872 inodes available. Want 99872. rc=0 - open/close 658 (time 1763346010.14 total 10.02 last 65.68) - open/close 1284 (time 1763346020.17 total 20.04 last 62.43) - open/close 1911 (time 1763346030.18 total 30.05 last 62.65) - open/close 2594 (time 1763346040.18 total 40.05 last 68.29) - open/close 3178 (time 1763346050.19 total 50.06 last 58.35) - open/close 3833 (time 1763346060.19 total 60.06 last 65.48) - open/close 4511 (time 1763346070.19 total 70.07 last 67.80) - open/close 5170 (time 1763346080.20 total 80.07 last 65.86) - open/close 5839 (time 1763346090.20 total 90.08 last 66.86) - open/close 6522 (time 1763346100.20 total 100.08 last 68.29) - open/close 7171 (time 1763346110.21 total 110.09 last 64.83) - open/close 7770 (time 1763346120.23 total 120.11 last 59.79) - open/close 8299 (time 1763346130.25 total 130.12 last 52.83) - open/close 8841 (time 1763346140.25 total 140.12 last 54.19) - open/close 9484 (time 1763346150.26 total 150.14 last 64.22) total: 10000 open/close in 157.80 seconds: 63.37 ops/second - unlinked 0 (time 1763346162 ; total 0 ; last 0) total: 10000 unlinks in 92 seconds: 108.695656 unlinks/second - open/close 676 (time 1763346268.87 total 10.01 last 67.50) - open/close 1317 (time 1763346278.87 total 20.02 last 64.10) - open/close 1971 (time 1763346288.88 total 30.02 last 65.36) - open/close 2638 (time 1763346298.89 total 40.03 last 66.63) - open/close 3312 (time 1763346308.91 total 50.06 last 67.24) - open/close 3918 (time 1763346318.91 total 60.06 last 60.60) - open/close 4530 (time 1763346328.92 total 70.06 last 61.16) - open/close 5192 (time 1763346338.92 total 80.07 last 66.18) - open/close 5888 (time 1763346348.93 total 90.07 last 69.55) - open/close 6554 (time 1763346358.93 total 100.08 last 66.58) - open/close 7188 (time 1763346368.94 total 110.08 last 63.39) - open/close 7902 (time 1763346378.95 total 120.09 last 71.29) - open/close 8566 (time 1763346388.97 total 130.11 last 66.27) - open/close 9188 (time 1763346398.97 total 140.12 last 62.19) - open/close 9840 (time 1763346408.98 total 150.12 last 65.17) total: 10000 open/close in 152.37 seconds: 65.63 ops/second - unlinked 0 (time 1763346414 ; total 0 ; last 0) total: 10000 unlinks in 83 seconds: 120.481926 unlinks/second - open/close 703 (time 1763346510.68 total 10.01 last 70.22) - open/close 1392 (time 1763346520.69 total 20.02 last 68.85) - open/close 2110 (time 1763346530.70 total 30.03 last 71.70) - open/close 2783 (time 1763346540.72 total 40.05 last 67.16) - open/close 3480 (time 1763346550.73 total 50.06 last 69.66) - open/close 4174 (time 1763346560.74 total 60.07 last 69.30) - open/close 4889 (time 1763346570.75 total 70.08 last 71.47) - open/close 5574 (time 1763346580.77 total 80.10 last 68.34) - open/close 6236 (time 1763346590.78 total 90.11 last 66.13) - open/close 6918 (time 1763346600.79 total 100.12 last 68.12) - open/close 7617 (time 1763346610.80 total 110.13 last 69.87) - open/close 8295 (time 1763346620.80 total 120.13 last 67.79) - open/close 8986 (time 1763346630.82 total 130.15 last 68.96) - open/close 9709 (time 1763346640.83 total 140.16 last 72.23) total: 10000 open/close in 143.73 seconds: 69.58 ops/second - unlinked 0 (time 1763346647 ; total 0 ; last 0) total: 10000 unlinks in 75 seconds: 133.333328 unlinks/second - open/close 790 (time 1763346735.27 total 10.01 last 78.91) - open/close 1474 (time 1763346745.27 total 20.02 last 68.36) - open/close 2158 (time 1763346755.28 total 30.02 last 68.36) - open/close 2859 (time 1763346765.29 total 40.03 last 70.04) - open/close 3584 (time 1763346775.30 total 50.04 last 72.43) - open/close 4323 (time 1763346785.31 total 60.05 last 73.81) - open/close 5066 (time 1763346795.32 total 70.06 last 74.23) - open/close 5764 (time 1763346805.32 total 80.06 last 69.78) - open/close 6460 (time 1763346815.33 total 90.07 last 69.56) - open/close 7154 (time 1763346825.34 total 100.09 last 69.30) - open/close 7867 (time 1763346835.34 total 110.09 last 71.30) - open/close 8597 (time 1763346845.35 total 120.09 last 72.97) - open/close 9237 (time 1763346855.35 total 130.10 last 63.95) - open/close 9934 (time 1763346865.36 total 140.10 last 69.66) total: 10000 open/close in 140.96 seconds: 70.94 ops/second - unlinked 0 (time 1763346869 ; total 0 ; last 0) total: 10000 unlinks in 81 seconds: 123.456787 unlinks/second - open/close 703 (time 1763346963.97 total 10.02 last 70.18) - open/close 1397 (time 1763346973.97 total 20.02 last 69.36) - open/close 2128 (time 1763346983.97 total 30.02 last 73.09) - open/close 2858 (time 1763346993.98 total 40.03 last 72.98) - open/close 3579 (time 1763347003.98 total 50.03 last 72.05) - open/close 4329 (time 1763347013.99 total 60.04 last 74.96) - open/close 5029 (time 1763347023.99 total 70.04 last 69.97) - open/close 5707 (time 1763347034.00 total 80.05 last 67.76) - open/close 6369 (time 1763347044.01 total 90.06 last 66.10) - open/close 7026 (time 1763347054.02 total 100.07 last 65.64) - open/close 7698 (time 1763347064.04 total 110.09 last 67.07) - open/close 8365 (time 1763347074.05 total 120.10 last 66.63) - open/close 9011 (time 1763347084.08 total 130.13 last 64.43) - open/close 9699 (time 1763347094.08 total 140.14 last 68.76) total: 10000 open/close in 145.55 seconds: 68.71 ops/second - unlinked 0 (time 1763347103 ; total 0 ; last 0) total: 10000 unlinks in 87 seconds: 114.942528 unlinks/second - open/close 674 (time 1763347205.08 total 10.01 last 67.36) - open/close 1301 (time 1763347215.10 total 20.03 last 62.58) - open/close 1907 (time 1763347225.10 total 30.03 last 60.58) - open/close 2455 (time 1763347235.11 total 40.03 last 54.78) - open/close 2950 (time 1763347245.11 total 50.04 last 49.47) - open/close 3623 (time 1763347255.11 total 60.04 last 67.29) - open/close 4282 (time 1763347265.12 total 70.04 last 65.88) - open/close 5003 (time 1763347275.12 total 80.05 last 72.08) - open/close 5667 (time 1763347285.12 total 90.05 last 66.36) - open/close 6335 (time 1763347295.13 total 100.06 last 66.76) - open/close 7029 (time 1763347305.13 total 110.06 last 69.37) - open/close 7674 (time 1763347315.14 total 120.06 last 64.48) - open/close 8301 (time 1763347325.15 total 130.08 last 62.62) - open/close 8985 (time 1763347335.16 total 140.09 last 68.34) - open/close 9612 (time 1763347345.17 total 150.10 last 62.64) total: 10000 open/close in 155.75 seconds: 64.21 ops/second - unlinked 0 (time 1763347355 ; total 0 ; last 0) total: 10000 unlinks in 89 seconds: 112.359550 unlinks/second - open/close 714 (time 1763347458.59 total 10.01 last 71.34) - open/close 1376 (time 1763347468.60 total 20.02 last 66.15) - open/close 2030 (time 1763347478.61 total 30.02 last 65.35) - open/close 2669 (time 1763347488.62 total 40.03 last 63.84) - open/close 3300 (time 1763347498.62 total 50.04 last 63.05) - open/close 3906 (time 1763347508.63 total 60.05 last 60.56) - open/close 4596 (time 1763347518.64 total 70.05 last 68.95) - open/close 5272 (time 1763347528.65 total 80.06 last 67.55) - open/close 5929 (time 1763347538.66 total 90.08 last 65.60) - open/close 6619 (time 1763347548.66 total 100.08 last 68.98) - open/close 7249 (time 1763347558.68 total 110.10 last 62.90) - open/close 7878 (time 1763347568.69 total 120.11 last 62.83) - open/close 8553 (time 1763347578.70 total 130.12 last 67.46) - open/close 9164 (time 1763347588.70 total 140.12 last 61.09) - open/close 9817 (time 1763347598.72 total 150.14 last 65.17) total: 10000 open/close in 152.58 seconds: 65.54 ops/second - unlinked 0 (time 1763347606 ; total 0 ; last 0) total: 10000 unlinks in 95 seconds: 105.263161 unlinks/second - open/close 581 (time 1763347716.49 total 10.02 last 57.97) - open/close 1219 (time 1763347726.50 total 20.03 last 63.73) - open/close 1880 (time 1763347736.51 total 30.04 last 66.07) - open/close 2527 (time 1763347746.52 total 40.05 last 64.63) - open/close 3225 (time 1763347756.53 total 50.06 last 69.74) - open/close 3839 (time 1763347766.53 total 60.07 last 61.35) - open/close 4452 (time 1763347776.53 total 70.07 last 61.30) - open/close 5078 (time 1763347786.55 total 80.08 last 62.51) - open/close 5710 (time 1763347796.55 total 90.09 last 63.17) - open/close 6363 (time 1763347806.56 total 100.09 last 65.27) - open/close 7008 (time 1763347816.57 total 110.11 last 64.41) - open/close 7677 (time 1763347826.59 total 120.12 last 66.79) - open/close 8305 (time 1763347836.60 total 130.13 last 62.73) - open/close 8968 (time 1763347846.61 total 140.14 last 66.25) - open/close 9543 (time 1763347856.61 total 150.14 last 57.48) total: 10000 open/close in 156.78 seconds: 63.78 ops/second - unlinked 0 (time 1763347867 ; total 0 ; last 0) total: 10000 unlinks in 90 seconds: 111.111115 unlinks/second - open/close 659 (time 1763347972.13 total 10.00 last 65.90) - open/close 1261 (time 1763347982.13 total 20.00 last 60.17) - open/close 1935 (time 1763347992.15 total 30.02 last 67.28) - open/close 2620 (time 1763348002.16 total 40.03 last 68.42) - open/close 3315 (time 1763348012.19 total 50.06 last 69.31) - open/close 4010 (time 1763348022.19 total 60.06 last 69.48) - open/close 4663 (time 1763348032.19 total 70.07 last 65.29) - open/close 5377 (time 1763348042.21 total 80.08 last 71.28) - open/close 6083 (time 1763348052.22 total 90.09 last 70.53) - open/close 6705 (time 1763348062.23 total 100.10 last 62.13) - open/close 7401 (time 1763348072.24 total 110.11 last 69.53) - open/close 8101 (time 1763348082.25 total 120.12 last 69.98) - open/close 8690 (time 1763348092.25 total 130.12 last 58.85) - open/close 9398 (time 1763348102.26 total 140.13 last 70.74) total: 10000 open/close in 149.25 seconds: 67.00 ops/second - unlinked 0 (time 1763348114 ; total 0 ; last 0) total: 10000 unlinks in 86 seconds: 116.279068 unlinks/second - open/close 731 (time 1763348214.20 total 10.01 last 72.99) - open/close 1419 (time 1763348224.21 total 20.02 last 68.77) - open/close 2089 (time 1763348234.21 total 30.02 last 67.00) - open/close 2754 (time 1763348244.21 total 40.02 last 66.48) - open/close 3422 (time 1763348254.22 total 50.04 last 66.70) - open/close 4122 (time 1763348264.23 total 60.05 last 69.92) - open/close 4880 (time 1763348274.24 total 70.05 last 75.78) - open/close 5545 (time 1763348284.24 total 80.05 last 66.47) - open/close 6170 (time 1763348294.24 total 90.06 last 62.49) - open/close 6829 (time 1763348304.24 total 100.06 last 65.88) - open/close 7474 (time 1763348314.26 total 110.07 last 64.42) - open/close 8164 (time 1763348324.27 total 120.08 last 68.90) - open/close 8832 (time 1763348334.28 total 130.09 last 66.75) - open/close 9509 (time 1763348344.29 total 140.10 last 67.65) total: 10000 open/close in 147.80 seconds: 67.66 ops/second - unlinked 0 (time 1763348356 ; total 0 ; last 0) total: 10000 unlinks in 88 seconds: 113.636360 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg152-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg152-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg152-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.201.152@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.201.152@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg152-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg152-server: oleg152-server.virtnet: executing set_default_debug -1 all pdsh@oleg152-client: oleg152-server: ssh exited with exit code 1 Started lustre-OST0000 oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg152-server: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg152-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg152-client.virtnet: -o user_xattr,flock 192.168.201.152@tcp:/lustre /mnt/lustre On OST0, 20325 used inodes rc=0 pdsh@oleg152-client: oleg152-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg152-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg152-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg152-server Permanent disk data: Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: mgsnode=192.168.201.152@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.201.152@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=98 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre:OST0000 checking for existing Lustre data: found Read previous values: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: sys.timeout=20 autodegrade=on mgsnode=192.168.201.152@tcp Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: sys.timeout=20 autodegrade=on mgsnode=192.168.201.152@tcp autodegrade=on lustre:svname=lustre-OST0000 Writing lustre-ost1/ost1 properties lustre:sys.timeout=20 lustre:autodegrade=on lustre:mgsnode=192.168.201.152@tcp lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg152-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg152-server: oleg152-server.virtnet: executing set_default_debug -1 all pdsh@oleg152-client: oleg152-server: ssh exited with exit code 1 Started lustre-OST0000 oleg152-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg152-server: oleg152-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg152-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec mount lustre on /mnt/lustre..... Starting client: oleg152-client.virtnet: -o user_xattr,flock 192.168.201.152@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg152-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg152-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg152-server stop mds service on oleg152-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg152-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg152-server' oleg152-server: oleg152-server.virtnet: executing unload_modules_local oleg152-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg152-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg152-client: oleg152-server: ssh exited with exit code 2 pdsh@oleg152-client: oleg152-client: ssh exited with exit code 2 pdsh@oleg152-client: oleg152-server: ssh exited with exit code 2 pdsh@oleg152-client: oleg152-client: ssh exited with exit code 2 PASS 69 (2641s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 22:02:57 (1763348577) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (3s) Stopping clients: oleg152-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg152-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg152-client: oleg152-server: ssh exited with exit code 2 oleg152-server: oleg152-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg152-server' oleg152-server: oleg152-server.virtnet: executing load_modules_local oleg152-server: Loading modules from /home/green/git/lustre-release/lustre oleg152-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg152-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg152-server: detected 4 online CPUs by sysfs oleg152-server: MODOPTS_LIBCFS= oleg152-server: Force libcfs to create 2 CPU partitions oleg152-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg152-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 3042 sec ========== 22:03:58 (1763348638) === conf-sanity: start cleanup 22:04:00 (1763348640) === === conf-sanity: finish cleanup 22:04:03 (1763348643) ===