-----============= acceptance-small: conf-sanity ============----- Sun Nov 16 17:48:16 EST 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg406-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32b 32c 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg406-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg406-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg406-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg406-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg406-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg406-server oleg406-server: oleg406-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg406-server' oleg406-server: oleg406-server.virtnet: executing load_modules_local oleg406-server: Loading modules from /home/green/git/lustre-release/lustre oleg406-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg406-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg406-server: detected 4 online CPUs by sysfs oleg406-server: MODOPTS_LIBCFS= oleg406-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg406-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg406-server: oleg406-server.virtnet: executing set_default_debug -1 all pdsh@oleg406-client: oleg406-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg406-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg406-server: oleg406-server.virtnet: executing set_default_debug -1 all pdsh@oleg406-client: oleg406-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg406-server: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg406-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg406-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg406-server stop mds service on oleg406-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg406-server == conf-sanity test 45: long unlink handling in ptlrpcd == 17:50:27 (1763333427) start mds service on oleg406-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg406-server: oleg406-server.virtnet: executing set_default_debug -1 all pdsh@oleg406-client: oleg406-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg406-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg406-server: oleg406-server.virtnet: executing set_default_debug -1 all pdsh@oleg406-client: oleg406-server: ssh exited with exit code 1 Started lustre-OST0000 oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg406-client.virtnet: -o user_xattr,flock 192.168.204.106@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg406-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg406-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg406-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg406-server: oleg406-server.virtnet: executing set_default_debug -1 all pdsh@oleg406-client: oleg406-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg406-client.virtnet: -o user_xattr,flock 192.168.204.106@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg406-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg406-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg406-server stop mds service on oleg406-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg406-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg406-server' oleg406-server: oleg406-server.virtnet: executing unload_modules_local oleg406-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg406-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg406-client: oleg406-client: ssh exited with exit code 2 pdsh@oleg406-client: oleg406-server: ssh exited with exit code 2 pdsh@oleg406-client: oleg406-client: ssh exited with exit code 2 pdsh@oleg406-client: oleg406-server: ssh exited with exit code 2 PASS 45 (190s) == conf-sanity test 69: replace an OST with the same index ========================================================== 17:53:37 (1763333617) start mds service on oleg406-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg406-server' oleg406-server: oleg406-server.virtnet: executing load_modules_local oleg406-server: Loading modules from /home/green/git/lustre-release/lustre oleg406-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg406-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg406-server: detected 4 online CPUs by sysfs oleg406-server: MODOPTS_LIBCFS= oleg406-server: Force libcfs to create 2 CPU partitions oleg406-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg406-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg406-server: oleg406-server.virtnet: executing set_default_debug -1 all pdsh@oleg406-client: oleg406-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg406-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg406-server: oleg406-server.virtnet: executing set_default_debug -1 all pdsh@oleg406-client: oleg406-server: ssh exited with exit code 1 Started lustre-OST0000 oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg406-client.virtnet: -o user_xattr,flock 192.168.204.106@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 11872 inodes available. Want 99872. rc=0 - open/close 695 (time 1763333690.25 total 10.01 last 69.45) - open/close 1357 (time 1763333700.26 total 20.02 last 66.13) - open/close 2025 (time 1763333710.27 total 30.02 last 66.76) - open/close 2687 (time 1763333720.28 total 40.03 last 66.16) - open/close 3368 (time 1763333730.30 total 50.05 last 67.94) - open/close 4035 (time 1763333740.31 total 60.06 last 66.64) - open/close 4629 (time 1763333750.32 total 70.08 last 59.32) - open/close 5308 (time 1763333760.33 total 80.09 last 67.82) - open/close 6070 (time 1763333770.34 total 90.09 last 76.16) - open/close 6818 (time 1763333780.34 total 100.10 last 74.78) - open/close 7583 (time 1763333790.34 total 110.10 last 76.49) - open/close 8366 (time 1763333800.35 total 120.10 last 78.27) - open/close 9107 (time 1763333810.36 total 130.11 last 74.02) - open/close 9851 (time 1763333820.36 total 140.12 last 74.38) total: 10000 open/close in 142.30 seconds: 70.27 ops/second - unlinked 0 (time 1763333825 ; total 0 ; last 0) total: 10000 unlinks in 88 seconds: 113.636360 unlinks/second - open/close 716 (time 1763333927.39 total 10.00 last 71.57) - open/close 1377 (time 1763333937.40 total 20.01 last 66.03) - open/close 2106 (time 1763333947.40 total 30.02 last 72.87) - open/close 2815 (time 1763333957.40 total 40.02 last 70.89) - open/close 3531 (time 1763333967.40 total 50.02 last 71.58) - open/close 4203 (time 1763333977.41 total 60.03 last 67.14) - open/close 4910 (time 1763333987.42 total 70.03 last 70.69) - open/close 5517 (time 1763333997.43 total 80.04 last 60.64) - open/close 6068 (time 1763334007.43 total 90.04 last 55.09) - open/close 6636 (time 1763334017.43 total 100.05 last 56.78) - open/close 7262 (time 1763334027.44 total 110.05 last 62.57) - open/close 7872 (time 1763334037.45 total 120.07 last 60.91) - open/close 8519 (time 1763334047.46 total 130.08 last 64.63) - open/close 9095 (time 1763334057.48 total 140.09 last 57.51) - open/close 9612 (time 1763334067.49 total 150.10 last 51.65) total: 10000 open/close in 157.00 seconds: 63.69 ops/second - unlinked 0 (time 1763334078 ; total 0 ; last 0) total: 10000 unlinks in 84 seconds: 119.047623 unlinks/second - open/close 673 (time 1763334176.67 total 10.00 last 67.27) - open/close 1361 (time 1763334186.67 total 20.01 last 68.78) - open/close 2096 (time 1763334196.68 total 30.02 last 73.42) - open/close 2958 (time 1763334206.69 total 40.02 last 86.14) - open/close 3764 (time 1763334216.71 total 50.04 last 80.44) - open/close 4524 (time 1763334226.72 total 60.06 last 75.90) - open/close 5278 (time 1763334236.73 total 70.06 last 75.36) - open/close 6067 (time 1763334246.73 total 80.07 last 78.87) - open/close 6814 (time 1763334256.73 total 90.07 last 74.68) - open/close 7531 (time 1763334266.74 total 100.08 last 71.65) - open/close 8250 (time 1763334276.74 total 110.08 last 71.89) - open/close 8943 (time 1763334286.75 total 120.09 last 69.23) - open/close 9785 (time 1763334296.76 total 130.10 last 84.11) total: 10000 open/close in 132.44 seconds: 75.50 ops/second - unlinked 0 (time 1763334301 ; total 0 ; last 0) total: 10000 unlinks in 75 seconds: 133.333328 unlinks/second - open/close 708 (time 1763334390.56 total 10.00 last 70.77) - open/close 1376 (time 1763334400.58 total 20.02 last 66.71) - open/close 2072 (time 1763334410.58 total 30.02 last 69.56) - open/close 2799 (time 1763334420.59 total 40.03 last 72.66) - open/close 3491 (time 1763334430.59 total 50.03 last 69.19) - open/close 4166 (time 1763334440.60 total 60.04 last 67.45) - open/close 4907 (time 1763334450.61 total 70.05 last 74.04) - open/close 5594 (time 1763334460.61 total 80.05 last 68.65) - open/close 6368 (time 1763334470.62 total 90.07 last 77.31) - open/close 7051 (time 1763334480.63 total 100.07 last 68.25) - open/close 7777 (time 1763334490.63 total 110.07 last 72.60) - open/close 8533 (time 1763334500.64 total 120.08 last 75.52) - open/close 9347 (time 1763334510.65 total 130.09 last 81.34) total: 10000 open/close in 138.08 seconds: 72.42 ops/second - unlinked 0 (time 1763334522 ; total 0 ; last 0) total: 10000 unlinks in 82 seconds: 121.951218 unlinks/second - open/close 655 (time 1763334619.56 total 10.00 last 65.48) - open/close 1390 (time 1763334629.56 total 20.01 last 73.45) - open/close 2094 (time 1763334639.57 total 30.02 last 70.36) - open/close 2814 (time 1763334649.58 total 40.03 last 71.91) - open/close 3546 (time 1763334659.58 total 50.03 last 73.19) - open/close 4267 (time 1763334669.59 total 60.03 last 72.08) - open/close 4969 (time 1763334679.61 total 70.06 last 70.04) - open/close 5678 (time 1763334689.62 total 80.07 last 70.83) - open/close 6351 (time 1763334699.62 total 90.07 last 67.28) - open/close 7016 (time 1763334709.63 total 100.08 last 66.42) - open/close 7677 (time 1763334719.64 total 110.09 last 66.04) - open/close 8314 (time 1763334729.65 total 120.10 last 63.66) - open/close 9003 (time 1763334739.66 total 130.11 last 68.84) - open/close 9686 (time 1763334749.66 total 140.11 last 68.28) total: 10000 open/close in 144.65 seconds: 69.13 ops/second - unlinked 0 (time 1763334758 ; total 0 ; last 0) total: 10000 unlinks in 83 seconds: 120.481926 unlinks/second - open/close 705 (time 1763334855.94 total 10.00 last 70.48) - open/close 1400 (time 1763334865.94 total 20.01 last 69.46) - open/close 2041 (time 1763334875.94 total 30.01 last 64.10) - open/close 2751 (time 1763334885.95 total 40.02 last 70.94) - open/close 3467 (time 1763334895.95 total 50.02 last 71.58) - open/close 4101 (time 1763334905.96 total 60.03 last 63.35) - open/close 4753 (time 1763334915.97 total 70.03 last 65.16) - open/close 5427 (time 1763334925.99 total 80.05 last 67.28) - open/close 6093 (time 1763334936.00 total 90.07 last 66.51) - open/close 6745 (time 1763334946.01 total 100.07 last 65.16) - open/close 7413 (time 1763334956.02 total 110.09 last 66.68) - open/close 8085 (time 1763334966.03 total 120.09 last 67.17) - open/close 8771 (time 1763334976.04 total 130.10 last 68.55) - open/close 9456 (time 1763334986.05 total 140.12 last 68.37) total: 10000 open/close in 148.05 seconds: 67.55 ops/second - unlinked 0 (time 1763334998 ; total 0 ; last 0) total: 10000 unlinks in 84 seconds: 119.047623 unlinks/second - open/close 706 (time 1763335096.74 total 10.01 last 70.51) - open/close 1382 (time 1763335106.74 total 20.01 last 67.60) - open/close 2132 (time 1763335116.74 total 30.02 last 74.96) - open/close 2731 (time 1763335126.76 total 40.03 last 59.82) - open/close 3362 (time 1763335136.77 total 50.05 last 63.01) - open/close 4064 (time 1763335146.79 total 60.06 last 70.08) - open/close 4779 (time 1763335156.80 total 70.08 last 71.42) - open/close 5516 (time 1763335166.81 total 80.09 last 73.61) - open/close 6223 (time 1763335176.83 total 90.11 last 70.57) - open/close 6897 (time 1763335186.83 total 100.11 last 67.40) - open/close 7584 (time 1763335196.83 total 110.11 last 68.70) - open/close 8319 (time 1763335206.84 total 120.12 last 73.40) - open/close 9031 (time 1763335216.85 total 130.13 last 71.14) - open/close 9755 (time 1763335226.86 total 140.14 last 72.32) total: 10000 open/close in 143.44 seconds: 69.72 ops/second - unlinked 0 (time 1763335234 ; total 0 ; last 0) total: 10000 unlinks in 84 seconds: 119.047623 unlinks/second - open/close 791 (time 1763335331.70 total 10.00 last 79.07) - open/close 1417 (time 1763335341.71 total 20.02 last 62.50) - open/close 2097 (time 1763335351.72 total 30.03 last 67.92) - open/close 2791 (time 1763335361.73 total 40.04 last 69.35) - open/close 3447 (time 1763335371.74 total 50.04 last 65.57) - open/close 4121 (time 1763335381.74 total 60.05 last 67.34) - open/close 4801 (time 1763335391.75 total 70.06 last 67.94) - open/close 5506 (time 1763335401.75 total 80.06 last 70.49) - open/close 6204 (time 1763335411.76 total 90.07 last 69.73) - open/close 6920 (time 1763335421.77 total 100.08 last 71.56) - open/close 7641 (time 1763335431.79 total 110.10 last 71.97) - open/close 8358 (time 1763335441.80 total 120.11 last 71.62) - open/close 9037 (time 1763335451.80 total 130.11 last 67.87) - open/close 9744 (time 1763335461.81 total 140.11 last 70.68) total: 10000 open/close in 143.98 seconds: 69.45 ops/second - unlinked 0 (time 1763335469 ; total 0 ; last 0) total: 10000 unlinks in 83 seconds: 120.481926 unlinks/second - open/close 669 (time 1763335566.44 total 10.00 last 66.90) - open/close 1375 (time 1763335576.44 total 20.00 last 70.59) - open/close 2063 (time 1763335586.44 total 30.00 last 68.80) - open/close 2787 (time 1763335596.45 total 40.01 last 72.37) - open/close 3479 (time 1763335606.45 total 50.01 last 69.19) - open/close 4192 (time 1763335616.46 total 60.02 last 71.23) - open/close 4884 (time 1763335626.47 total 70.03 last 69.12) - open/close 5567 (time 1763335636.49 total 80.05 last 68.16) - open/close 6219 (time 1763335646.50 total 90.06 last 65.10) - open/close 6894 (time 1763335656.52 total 100.08 last 67.41) - open/close 7539 (time 1763335666.52 total 110.08 last 64.47) - open/close 8209 (time 1763335676.53 total 120.09 last 66.96) - open/close 8884 (time 1763335686.53 total 130.09 last 67.45) - open/close 9599 (time 1763335696.53 total 140.10 last 71.50) total: 10000 open/close in 145.70 seconds: 68.63 ops/second - unlinked 0 (time 1763335705 ; total 0 ; last 0) total: 10000 unlinks in 80 seconds: 125.000000 unlinks/second - open/close 683 (time 1763335799.71 total 10.01 last 68.26) - open/close 1375 (time 1763335809.72 total 20.02 last 69.14) - open/close 2057 (time 1763335819.74 total 30.03 last 68.10) - open/close 2789 (time 1763335829.74 total 40.03 last 73.17) - open/close 3539 (time 1763335839.75 total 50.04 last 74.95) - open/close 4271 (time 1763335849.75 total 60.04 last 73.19) - open/close 4986 (time 1763335859.75 total 70.04 last 71.47) - open/close 5683 (time 1763335869.76 total 80.05 last 69.64) - open/close 6346 (time 1763335879.77 total 90.06 last 66.26) - open/close 7009 (time 1763335889.79 total 100.08 last 66.17) - open/close 7674 (time 1763335899.80 total 110.09 last 66.42) - open/close 8337 (time 1763335909.81 total 120.10 last 66.22) - open/close 9049 (time 1763335919.81 total 130.10 last 71.19) - open/close 9748 (time 1763335929.81 total 140.10 last 69.90) total: 10000 open/close in 143.47 seconds: 69.70 ops/second - unlinked 0 (time 1763335937 ; total 0 ; last 0) total: 10000 unlinks in 88 seconds: 113.636360 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg406-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg406-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg406-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.204.106@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.204.106@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg406-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg406-server: oleg406-server.virtnet: executing set_default_debug -1 all pdsh@oleg406-client: oleg406-server: ssh exited with exit code 1 Started lustre-OST0000 oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg406-server: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg406-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg406-client.virtnet: -o user_xattr,flock 192.168.204.106@tcp:/lustre /mnt/lustre On OST0, 20325 used inodes rc=0 pdsh@oleg406-client: oleg406-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg406-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg406-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg406-server Permanent disk data: Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: mgsnode=192.168.204.106@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.204.106@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=98 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre:OST0000 checking for existing Lustre data: found Read previous values: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: sys.timeout=20 autodegrade=on mgsnode=192.168.204.106@tcp Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: sys.timeout=20 autodegrade=on mgsnode=192.168.204.106@tcp autodegrade=on lustre:svname=lustre-OST0000 Writing lustre-ost1/ost1 properties lustre:sys.timeout=20 lustre:autodegrade=on lustre:mgsnode=192.168.204.106@tcp lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg406-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg406-server: oleg406-server.virtnet: executing set_default_debug -1 all pdsh@oleg406-client: oleg406-server: ssh exited with exit code 1 Started lustre-OST0000 oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg406-server: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg406-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg406-client.virtnet: -o user_xattr,flock 192.168.204.106@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg406-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg406-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg406-server stop mds service on oleg406-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg406-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg406-server' oleg406-server: oleg406-server.virtnet: executing unload_modules_local oleg406-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg406-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg406-client: oleg406-client: ssh exited with exit code 2 pdsh@oleg406-client: oleg406-server: ssh exited with exit code 2 pdsh@oleg406-client: oleg406-server: ssh exited with exit code 2 pdsh@oleg406-client: oleg406-client: ssh exited with exit code 2 PASS 69 (2546s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 18:36:04 (1763336164) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (3s) Stopping clients: oleg406-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg406-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg406-client: oleg406-server: ssh exited with exit code 2 oleg406-server: oleg406-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg406-server' oleg406-server: oleg406-server.virtnet: executing load_modules_local oleg406-server: Loading modules from /home/green/git/lustre-release/lustre oleg406-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg406-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg406-server: detected 4 online CPUs by sysfs oleg406-server: MODOPTS_LIBCFS= oleg406-server: Force libcfs to create 2 CPU partitions oleg406-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg406-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 2943 sec ========== 18:37:20 (1763336240) === conf-sanity: start cleanup 18:37:22 (1763336242) === === conf-sanity: finish cleanup 18:37:26 (1763336246) ===