-----============= acceptance-small: conf-sanity ============----- Sun Nov 16 22:01:08 EST 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg157-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32b 32c 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg157-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg157-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg157-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg157-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg157-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg157-server oleg157-server: oleg157-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg157-server' oleg157-server: oleg157-server.virtnet: executing load_modules_local oleg157-server: Loading modules from /home/green/git/lustre-release/lustre oleg157-server: detected 4 online CPUs by sysfs oleg157-server: MODOPTS_LIBCFS= oleg157-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg157-server: Force libcfs to create 2 CPU partitions oleg157-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg157-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg157-server: oleg157-server.virtnet: executing set_default_debug -1 all pdsh@oleg157-client: oleg157-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg157-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg157-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg157-server: oleg157-server.virtnet: executing set_default_debug -1 all pdsh@oleg157-client: oleg157-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg157-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg157-server: oleg157-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg157-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg157-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg157-server stop mds service on oleg157-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg157-server == conf-sanity test 45: long unlink handling in ptlrpcd == 22:03:45 (1763348625) start mds service on oleg157-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg157-server: oleg157-server.virtnet: executing set_default_debug -1 all pdsh@oleg157-client: oleg157-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg157-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg157-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg157-server: oleg157-server.virtnet: executing set_default_debug -1 all pdsh@oleg157-client: oleg157-server: ssh exited with exit code 1 Started lustre-OST0000 oleg157-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg157-client.virtnet: -o user_xattr,flock 192.168.201.157@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg157-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg157-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg157-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg157-server: oleg157-server.virtnet: executing set_default_debug -1 all pdsh@oleg157-client: oleg157-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg157-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg157-client.virtnet: -o user_xattr,flock 192.168.201.157@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg157-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg157-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg157-server stop mds service on oleg157-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg157-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg157-server' oleg157-server: oleg157-server.virtnet: executing unload_modules_local oleg157-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg157-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg157-client: oleg157-server: ssh exited with exit code 2 pdsh@oleg157-client: oleg157-client: ssh exited with exit code 2 pdsh@oleg157-client: oleg157-client: ssh exited with exit code 2 pdsh@oleg157-client: oleg157-server: ssh exited with exit code 2 PASS 45 (199s) == conf-sanity test 69: replace an OST with the same index ========================================================== 22:07:04 (1763348824) start mds service on oleg157-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg157-server' oleg157-server: oleg157-server.virtnet: executing load_modules_local oleg157-server: Loading modules from /home/green/git/lustre-release/lustre oleg157-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg157-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg157-server: detected 4 online CPUs by sysfs oleg157-server: MODOPTS_LIBCFS= oleg157-server: Force libcfs to create 2 CPU partitions oleg157-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg157-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg157-server: oleg157-server.virtnet: executing set_default_debug -1 all pdsh@oleg157-client: oleg157-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg157-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg157-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg157-server: oleg157-server.virtnet: executing set_default_debug -1 all pdsh@oleg157-client: oleg157-server: ssh exited with exit code 1 Started lustre-OST0000 oleg157-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg157-client.virtnet: -o user_xattr,flock 192.168.201.157@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 11872 inodes available. Want 99872. rc=0 - open/close 685 (time 1763348903.98 total 10.01 last 68.45) - open/close 1362 (time 1763348913.98 total 20.01 last 67.66) - open/close 2020 (time 1763348923.99 total 30.02 last 65.75) - open/close 2648 (time 1763348934.00 total 40.03 last 62.74) - open/close 3343 (time 1763348944.00 total 50.03 last 69.49) - open/close 4048 (time 1763348954.02 total 60.05 last 70.37) - open/close 4694 (time 1763348964.02 total 70.05 last 64.59) - open/close 5414 (time 1763348974.03 total 80.06 last 71.93) - open/close 6101 (time 1763348984.05 total 90.08 last 68.60) - open/close 6800 (time 1763348994.05 total 100.08 last 69.87) - open/close 7493 (time 1763349004.05 total 110.08 last 69.29) - open/close 8283 (time 1763349014.06 total 120.09 last 78.94) - open/close 9046 (time 1763349024.07 total 130.10 last 76.24) - open/close 9799 (time 1763349034.08 total 140.11 last 75.22) total: 10000 open/close in 142.93 seconds: 69.97 ops/second - unlinked 0 (time 1763349040 ; total 0 ; last 0) total: 10000 unlinks in 77 seconds: 129.870132 unlinks/second - open/close 783 (time 1763349130.85 total 10.00 last 78.30) - open/close 1443 (time 1763349140.86 total 20.00 last 65.98) - open/close 2164 (time 1763349150.87 total 30.01 last 72.03) - open/close 2805 (time 1763349160.87 total 40.01 last 64.10) - open/close 3561 (time 1763349170.88 total 50.02 last 75.53) - open/close 4313 (time 1763349180.88 total 60.03 last 75.17) - open/close 5076 (time 1763349190.89 total 70.04 last 76.23) - open/close 5712 (time 1763349200.89 total 80.04 last 63.57) - open/close 6493 (time 1763349210.91 total 90.06 last 77.99) - open/close 7166 (time 1763349220.92 total 100.07 last 67.23) - open/close 7813 (time 1763349230.93 total 110.07 last 64.66) - open/close 8475 (time 1763349240.93 total 120.08 last 66.15) - open/close 9132 (time 1763349250.95 total 130.10 last 65.60) - open/close 9839 (time 1763349260.96 total 140.11 last 70.59) total: 10000 open/close in 142.36 seconds: 70.24 ops/second - unlinked 0 (time 1763349266 ; total 0 ; last 0) total: 10000 unlinks in 80 seconds: 125.000000 unlinks/second - open/close 660 (time 1763349360.48 total 10.00 last 65.98) - open/close 1341 (time 1763349370.50 total 20.02 last 67.99) - open/close 2025 (time 1763349380.51 total 30.03 last 68.34) - open/close 2661 (time 1763349390.51 total 40.03 last 63.59) - open/close 3410 (time 1763349400.51 total 50.03 last 74.89) - open/close 4142 (time 1763349410.52 total 60.04 last 73.17) - open/close 4816 (time 1763349420.52 total 70.04 last 67.36) - open/close 5437 (time 1763349430.53 total 80.05 last 62.04) - open/close 6092 (time 1763349440.53 total 90.05 last 65.48) - open/close 6762 (time 1763349450.55 total 100.07 last 66.90) - open/close 7382 (time 1763349460.56 total 110.08 last 61.92) - open/close 8081 (time 1763349470.56 total 120.08 last 69.89) - open/close 8749 (time 1763349480.57 total 130.09 last 66.77) - open/close 9512 (time 1763349490.57 total 140.09 last 76.27) total: 10000 open/close in 146.72 seconds: 68.16 ops/second - unlinked 0 (time 1763349501 ; total 0 ; last 0) total: 10000 unlinks in 85 seconds: 117.647057 unlinks/second - open/close 649 (time 1763349600.82 total 10.00 last 64.89) - open/close 1312 (time 1763349610.83 total 20.01 last 66.24) - open/close 1974 (time 1763349620.83 total 30.01 last 66.17) - open/close 2653 (time 1763349630.84 total 40.02 last 67.83) - open/close 3339 (time 1763349640.85 total 50.03 last 68.54) - open/close 4042 (time 1763349650.86 total 60.04 last 70.25) - open/close 4712 (time 1763349660.86 total 70.04 last 66.99) - open/close 5390 (time 1763349670.86 total 80.05 last 67.78) - open/close 6110 (time 1763349680.87 total 90.05 last 71.99) - open/close 6820 (time 1763349690.88 total 100.06 last 70.90) - open/close 7556 (time 1763349700.88 total 110.06 last 73.58) - open/close 8287 (time 1763349710.89 total 120.07 last 73.05) - open/close 9069 (time 1763349720.89 total 130.07 last 78.20) - open/close 9810 (time 1763349730.90 total 140.08 last 74.03) total: 10000 open/close in 142.75 seconds: 70.05 ops/second - unlinked 0 (time 1763349736 ; total 0 ; last 0) total: 10000 unlinks in 76 seconds: 131.578949 unlinks/second - open/close 731 (time 1763349826.61 total 10.00 last 73.07) - open/close 1531 (time 1763349836.62 total 20.01 last 79.97) - open/close 2207 (time 1763349846.62 total 30.01 last 67.59) - open/close 2821 (time 1763349856.63 total 40.02 last 61.33) - open/close 3469 (time 1763349866.64 total 50.03 last 64.76) - open/close 4092 (time 1763349876.64 total 60.03 last 62.27) - open/close 4740 (time 1763349886.67 total 70.05 last 64.65) - open/close 5394 (time 1763349896.67 total 80.06 last 65.36) - open/close 6070 (time 1763349906.68 total 90.07 last 67.54) - open/close 6757 (time 1763349916.68 total 100.07 last 68.67) - open/close 7471 (time 1763349926.70 total 110.09 last 71.31) - open/close 8172 (time 1763349936.70 total 120.09 last 70.06) - open/close 8877 (time 1763349946.71 total 130.10 last 70.44) - open/close 9536 (time 1763349956.72 total 140.11 last 65.85) total: 10000 open/close in 146.50 seconds: 68.26 ops/second - unlinked 0 (time 1763349966 ; total 0 ; last 0) total: 10000 unlinks in 82 seconds: 121.951218 unlinks/second - open/close 733 (time 1763350061.95 total 10.01 last 73.21) - open/close 1406 (time 1763350071.95 total 20.01 last 67.29) - open/close 2142 (time 1763350081.96 total 30.02 last 73.52) - open/close 2897 (time 1763350091.98 total 40.04 last 75.35) - open/close 3615 (time 1763350101.98 total 50.04 last 71.80) - open/close 4312 (time 1763350111.98 total 60.05 last 69.69) - open/close 5022 (time 1763350121.99 total 70.06 last 70.94) - open/close 5771 (time 1763350132.01 total 80.08 last 74.71) - open/close 6502 (time 1763350142.02 total 90.09 last 73.06) - open/close 7207 (time 1763350152.02 total 100.09 last 70.49) - open/close 7931 (time 1763350162.03 total 110.10 last 72.31) - open/close 8671 (time 1763350172.04 total 120.11 last 73.96) - open/close 9385 (time 1763350182.05 total 130.11 last 71.35) total: 10000 open/close in 138.90 seconds: 71.99 ops/second - unlinked 0 (time 1763350195 ; total 1 ; last 1) total: 10000 unlinks in 83 seconds: 120.481926 unlinks/second - open/close 750 (time 1763350290.97 total 10.01 last 74.90) - open/close 1373 (time 1763350300.97 total 20.02 last 62.28) - open/close 2035 (time 1763350310.98 total 30.03 last 66.09) - open/close 2741 (time 1763350320.99 total 40.04 last 70.53) - open/close 3464 (time 1763350331.01 total 50.06 last 72.20) - open/close 4189 (time 1763350341.01 total 60.06 last 72.48) - open/close 4932 (time 1763350351.02 total 70.07 last 74.20) - open/close 5655 (time 1763350361.03 total 80.08 last 72.25) - open/close 6403 (time 1763350371.04 total 90.09 last 74.75) - open/close 7170 (time 1763350381.04 total 100.09 last 76.70) - open/close 7848 (time 1763350391.05 total 110.10 last 67.74) - open/close 8606 (time 1763350401.05 total 120.10 last 75.79) - open/close 9318 (time 1763350411.05 total 130.10 last 71.19) total: 10000 open/close in 139.82 seconds: 71.52 ops/second - unlinked 0 (time 1763350424 ; total 0 ; last 0) total: 10000 unlinks in 83 seconds: 120.481926 unlinks/second - open/close 688 (time 1763350521.45 total 10.01 last 68.76) - open/close 1359 (time 1763350531.46 total 20.01 last 67.07) - open/close 2089 (time 1763350541.46 total 30.01 last 72.96) - open/close 2822 (time 1763350551.47 total 40.03 last 73.22) - open/close 3490 (time 1763350561.48 total 50.03 last 66.77) - open/close 4175 (time 1763350571.49 total 60.04 last 68.44) - open/close 4860 (time 1763350581.50 total 70.06 last 68.38) - open/close 5548 (time 1763350591.52 total 80.07 last 68.71) - open/close 6231 (time 1763350601.52 total 90.08 last 68.24) - open/close 6868 (time 1763350611.53 total 100.09 last 63.64) - open/close 7504 (time 1763350621.55 total 110.10 last 63.50) - open/close 8111 (time 1763350631.56 total 120.12 last 60.62) - open/close 8675 (time 1763350641.57 total 130.12 last 56.39) - open/close 9216 (time 1763350651.57 total 140.12 last 54.09) - open/close 9816 (time 1763350661.58 total 150.14 last 59.90) total: 10000 open/close in 153.28 seconds: 65.24 ops/second - unlinked 0 (time 1763350669 ; total 0 ; last 0) total: 10000 unlinks in 91 seconds: 109.890106 unlinks/second - open/close 625 (time 1763350775.55 total 10.00 last 62.49) - open/close 1263 (time 1763350785.56 total 20.01 last 63.74) - open/close 1926 (time 1763350795.57 total 30.02 last 66.21) - open/close 2569 (time 1763350805.59 total 40.04 last 64.21) - open/close 3188 (time 1763350815.59 total 50.04 last 61.88) - open/close 3827 (time 1763350825.60 total 60.05 last 63.84) - open/close 4396 (time 1763350835.62 total 70.07 last 56.80) - open/close 5007 (time 1763350845.63 total 80.08 last 61.02) - open/close 5641 (time 1763350855.64 total 90.09 last 63.34) - open/close 6307 (time 1763350865.65 total 100.10 last 66.55) - open/close 6897 (time 1763350875.66 total 110.11 last 58.93) - open/close 7525 (time 1763350885.67 total 120.12 last 62.76) - open/close 8103 (time 1763350895.69 total 130.14 last 57.69) - open/close 8687 (time 1763350905.69 total 140.14 last 58.38) - open/close 9307 (time 1763350915.70 total 150.15 last 61.97) - open/close 9926 (time 1763350925.71 total 160.16 last 61.79) total: 10000 open/close in 161.52 seconds: 61.91 ops/second - unlinked 0 (time 1763350931 ; total 0 ; last 0) total: 10000 unlinks in 90 seconds: 111.111115 unlinks/second - open/close 615 (time 1763351036.88 total 10.01 last 61.41) - open/close 1201 (time 1763351046.88 total 20.02 last 58.57) - open/close 1836 (time 1763351056.90 total 30.04 last 63.38) - open/close 2487 (time 1763351066.90 total 40.04 last 65.10) - open/close 3156 (time 1763351076.92 total 50.05 last 66.81) - open/close 3809 (time 1763351086.92 total 60.05 last 65.30) - open/close 4454 (time 1763351096.94 total 70.08 last 64.35) - open/close 5068 (time 1763351106.95 total 80.08 last 61.35) - open/close 5689 (time 1763351116.95 total 90.09 last 62.08) - open/close 6321 (time 1763351126.96 total 100.10 last 63.12) - open/close 6956 (time 1763351136.97 total 110.10 last 63.48) - open/close 7616 (time 1763351146.98 total 120.11 last 65.93) - open/close 8239 (time 1763351156.99 total 130.13 last 62.21) - open/close 8861 (time 1763351167.01 total 140.15 last 62.08) - open/close 9507 (time 1763351177.02 total 150.15 last 64.54) total: 10000 open/close in 158.48 seconds: 63.10 ops/second - unlinked 0 (time 1763351189 ; total 0 ; last 0) total: 10000 unlinks in 94 seconds: 106.382980 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg157-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg157-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg157-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.201.157@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.201.157@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg157-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg157-server: oleg157-server.virtnet: executing set_default_debug -1 all pdsh@oleg157-client: oleg157-server: ssh exited with exit code 1 Started lustre-OST0000 oleg157-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg157-server: oleg157-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg157-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg157-client.virtnet: -o user_xattr,flock 192.168.201.157@tcp:/lustre /mnt/lustre On OST0, 20325 used inodes rc=0 pdsh@oleg157-client: oleg157-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg157-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg157-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg157-server Permanent disk data: Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: mgsnode=192.168.201.157@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.201.157@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=98 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre:OST0000 checking for existing Lustre data: found Read previous values: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: autodegrade=on sys.timeout=20 mgsnode=192.168.201.157@tcp Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: autodegrade=on sys.timeout=20 mgsnode=192.168.201.157@tcp autodegrade=on lustre:svname=lustre-OST0000 Writing lustre-ost1/ost1 properties lustre:autodegrade=on lustre:sys.timeout=20 lustre:mgsnode=192.168.201.157@tcp lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg157-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg157-server: oleg157-server.virtnet: executing set_default_debug -1 all pdsh@oleg157-client: oleg157-server: ssh exited with exit code 1 Started lustre-OST0000 oleg157-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg157-server: oleg157-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg157-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg157-client.virtnet: -o user_xattr,flock 192.168.201.157@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg157-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg157-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg157-server stop mds service on oleg157-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg157-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg157-server' oleg157-server: oleg157-server.virtnet: executing unload_modules_local oleg157-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg157-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg157-client: oleg157-client: ssh exited with exit code 2 pdsh@oleg157-client: oleg157-server: ssh exited with exit code 2 pdsh@oleg157-client: oleg157-client: ssh exited with exit code 2 pdsh@oleg157-client: oleg157-server: ssh exited with exit code 2 PASS 69 (2583s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 22:50:08 (1763351408) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (3s) Stopping clients: oleg157-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg157-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg157-client: oleg157-server: ssh exited with exit code 2 oleg157-server: oleg157-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg157-server' oleg157-server: oleg157-server.virtnet: executing load_modules_local oleg157-server: Loading modules from /home/green/git/lustre-release/lustre oleg157-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg157-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg157-server: detected 4 online CPUs by sysfs oleg157-server: MODOPTS_LIBCFS= oleg157-server: Force libcfs to create 2 CPU partitions oleg157-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg157-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 3009 sec ========== 22:51:18 (1763351478) === conf-sanity: start cleanup 22:51:19 (1763351479) === === conf-sanity: finish cleanup 22:51:22 (1763351482) ===