-----============= acceptance-small: conf-sanity ============----- Sun Nov 16 18:40:38 EST 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg447-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32b 32c 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg447-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg447-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg447-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg447-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg447-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg447-server oleg447-server: oleg447-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg447-server' oleg447-server: oleg447-server.virtnet: executing load_modules_local oleg447-server: Loading modules from /home/green/git/lustre-release/lustre oleg447-server: detected 4 online CPUs by sysfs oleg447-server: MODOPTS_LIBCFS= oleg447-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg447-server: Force libcfs to create 2 CPU partitions oleg447-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg447-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg447-server: oleg447-server.virtnet: executing set_default_debug -1 all pdsh@oleg447-client: oleg447-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg447-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg447-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg447-server: oleg447-server.virtnet: executing set_default_debug -1 all pdsh@oleg447-client: oleg447-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg447-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg447-server: oleg447-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg447-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg447-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg447-server stop mds service on oleg447-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg447-server == conf-sanity test 45: long unlink handling in ptlrpcd == 18:42:59 (1763336579) start mds service on oleg447-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg447-server: oleg447-server.virtnet: executing set_default_debug -1 all pdsh@oleg447-client: oleg447-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg447-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg447-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg447-server: oleg447-server.virtnet: executing set_default_debug -1 all pdsh@oleg447-client: oleg447-server: ssh exited with exit code 1 Started lustre-OST0000 oleg447-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg447-client.virtnet: -o user_xattr,flock 192.168.204.147@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg447-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg447-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg447-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg447-server: oleg447-server.virtnet: executing set_default_debug -1 all pdsh@oleg447-client: oleg447-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg447-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg447-client.virtnet: -o user_xattr,flock 192.168.204.147@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg447-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg447-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg447-server stop mds service on oleg447-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg447-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg447-server' oleg447-server: oleg447-server.virtnet: executing unload_modules_local oleg447-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg447-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg447-client: oleg447-client: ssh exited with exit code 2 pdsh@oleg447-client: oleg447-server: ssh exited with exit code 2 pdsh@oleg447-client: oleg447-client: ssh exited with exit code 2 pdsh@oleg447-client: oleg447-server: ssh exited with exit code 2 PASS 45 (194s) == conf-sanity test 69: replace an OST with the same index ========================================================== 18:46:13 (1763336773) start mds service on oleg447-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg447-server' oleg447-server: oleg447-server.virtnet: executing load_modules_local oleg447-server: Loading modules from /home/green/git/lustre-release/lustre oleg447-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg447-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg447-server: detected 4 online CPUs by sysfs oleg447-server: MODOPTS_LIBCFS= oleg447-server: Force libcfs to create 2 CPU partitions oleg447-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg447-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg447-server: oleg447-server.virtnet: executing set_default_debug -1 all pdsh@oleg447-client: oleg447-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg447-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg447-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg447-server: oleg447-server.virtnet: executing set_default_debug -1 all pdsh@oleg447-client: oleg447-server: ssh exited with exit code 1 Started lustre-OST0000 oleg447-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg447-client.virtnet: -o user_xattr,flock 192.168.204.147@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 11872 inodes available. Want 99872. rc=0 - open/close 711 (time 1763336847.43 total 10.01 last 71.02) - open/close 1453 (time 1763336857.43 total 20.01 last 74.19) - open/close 2141 (time 1763336867.44 total 30.03 last 68.71) - open/close 2864 (time 1763336877.45 total 40.04 last 72.22) - open/close 3601 (time 1763336887.46 total 50.05 last 73.63) - open/close 4192 (time 1763336897.47 total 60.05 last 59.09) - open/close 4882 (time 1763336907.47 total 70.06 last 68.95) - open/close 5578 (time 1763336917.49 total 80.07 last 69.50) - open/close 6245 (time 1763336927.50 total 90.08 last 66.64) - open/close 6895 (time 1763336937.51 total 100.09 last 64.92) - open/close 7581 (time 1763336947.51 total 110.10 last 68.56) - open/close 8272 (time 1763336957.53 total 120.12 last 68.97) - open/close 8849 (time 1763336967.55 total 130.13 last 57.63) - open/close 9479 (time 1763336977.55 total 140.14 last 62.95) total: 10000 open/close in 147.42 seconds: 67.84 ops/second - unlinked 0 (time 1763336989 ; total 0 ; last 0) total: 10000 unlinks in 85 seconds: 117.647057 unlinks/second - open/close 678 (time 1763337088.02 total 10.00 last 67.80) - open/close 1397 (time 1763337098.02 total 20.00 last 71.89) - open/close 2096 (time 1763337108.03 total 30.00 last 69.88) - open/close 2803 (time 1763337118.03 total 40.00 last 70.70) - open/close 3503 (time 1763337128.03 total 50.01 last 69.99) - open/close 4182 (time 1763337138.03 total 60.01 last 67.86) - open/close 4872 (time 1763337148.06 total 70.03 last 68.84) - open/close 5517 (time 1763337158.06 total 80.04 last 64.49) - open/close 6188 (time 1763337168.07 total 90.04 last 67.06) - open/close 6859 (time 1763337178.07 total 100.05 last 67.08) - open/close 7547 (time 1763337188.08 total 110.06 last 68.72) - open/close 8238 (time 1763337198.08 total 120.06 last 69.08) - open/close 8965 (time 1763337208.09 total 130.07 last 72.66) - open/close 9687 (time 1763337218.11 total 140.09 last 72.06) total: 10000 open/close in 144.68 seconds: 69.12 ops/second - unlinked 0 (time 1763337226 ; total 0 ; last 0) total: 10000 unlinks in 82 seconds: 121.951218 unlinks/second - open/close 704 (time 1763337321.97 total 10.01 last 70.31) - open/close 1345 (time 1763337331.97 total 20.02 last 64.06) - open/close 2038 (time 1763337341.98 total 30.03 last 69.24) - open/close 2670 (time 1763337351.99 total 40.04 last 63.13) - open/close 3384 (time 1763337361.99 total 50.04 last 71.39) - open/close 4112 (time 1763337372.01 total 60.05 last 72.70) - open/close 4843 (time 1763337382.01 total 70.06 last 73.05) - open/close 5580 (time 1763337392.04 total 80.08 last 73.52) - open/close 6283 (time 1763337402.05 total 90.09 last 70.24) - open/close 6979 (time 1763337412.05 total 100.10 last 69.57) - open/close 7683 (time 1763337422.06 total 110.10 last 70.37) - open/close 8442 (time 1763337432.06 total 120.11 last 75.86) - open/close 9042 (time 1763337442.07 total 130.12 last 59.95) - open/close 9687 (time 1763337452.07 total 140.12 last 64.49) total: 10000 open/close in 144.76 seconds: 69.08 ops/second - unlinked 0 (time 1763337460 ; total 0 ; last 0) total: 10000 unlinks in 83 seconds: 120.481926 unlinks/second - open/close 698 (time 1763337558.22 total 10.01 last 69.73) - open/close 1429 (time 1763337568.23 total 20.02 last 73.02) - open/close 2160 (time 1763337578.24 total 30.03 last 73.00) - open/close 2867 (time 1763337588.25 total 40.04 last 70.67) - open/close 3589 (time 1763337598.26 total 50.05 last 72.11) - open/close 4252 (time 1763337608.26 total 60.05 last 66.28) - open/close 4909 (time 1763337618.26 total 70.05 last 65.69) - open/close 5613 (time 1763337628.26 total 80.05 last 70.40) - open/close 6295 (time 1763337638.26 total 90.06 last 68.20) - open/close 7013 (time 1763337648.27 total 100.06 last 71.75) - open/close 7727 (time 1763337658.28 total 110.07 last 71.34) - open/close 8420 (time 1763337668.29 total 120.08 last 69.24) - open/close 9092 (time 1763337678.30 total 130.10 last 67.08) - open/close 9795 (time 1763337688.31 total 140.10 last 70.28) total: 10000 open/close in 142.97 seconds: 69.94 ops/second - unlinked 0 (time 1763337695 ; total 0 ; last 0) total: 10000 unlinks in 82 seconds: 121.951218 unlinks/second - open/close 705 (time 1763337791.47 total 10.01 last 70.41) - open/close 1414 (time 1763337801.47 total 20.02 last 70.87) - open/close 2077 (time 1763337811.48 total 30.02 last 66.29) - open/close 2783 (time 1763337821.49 total 40.03 last 70.53) - open/close 3425 (time 1763337831.49 total 50.04 last 64.14) - open/close 4096 (time 1763337841.50 total 60.04 last 67.09) - open/close 4808 (time 1763337851.50 total 70.05 last 71.15) - open/close 5498 (time 1763337861.51 total 80.05 last 68.95) - open/close 6220 (time 1763337871.52 total 90.06 last 72.12) - open/close 6899 (time 1763337881.53 total 100.07 last 67.85) - open/close 7592 (time 1763337891.53 total 110.08 last 69.26) - open/close 8270 (time 1763337901.55 total 120.09 last 67.70) - open/close 9030 (time 1763337911.56 total 130.10 last 75.93) - open/close 9737 (time 1763337921.56 total 140.10 last 70.69) total: 10000 open/close in 143.98 seconds: 69.45 ops/second - unlinked 0 (time 1763337929 ; total 0 ; last 0) total: 10000 unlinks in 86 seconds: 116.279068 unlinks/second - open/close 659 (time 1763338029.20 total 10.01 last 65.84) - open/close 1295 (time 1763338039.20 total 20.01 last 63.57) - open/close 1978 (time 1763338049.20 total 30.02 last 68.30) - open/close 2644 (time 1763338059.21 total 40.02 last 66.59) - open/close 3347 (time 1763338069.22 total 50.03 last 70.18) - open/close 3976 (time 1763338079.23 total 60.04 last 62.84) - open/close 4685 (time 1763338089.24 total 70.05 last 70.85) - open/close 5382 (time 1763338099.24 total 80.05 last 69.69) - open/close 6039 (time 1763338109.25 total 90.06 last 65.66) - open/close 6817 (time 1763338119.28 total 100.09 last 77.55) - open/close 7519 (time 1763338129.29 total 110.10 last 70.14) - open/close 8251 (time 1763338139.30 total 120.11 last 73.13) - open/close 8894 (time 1763338149.30 total 130.11 last 64.29) - open/close 9532 (time 1763338159.30 total 140.11 last 63.80) total: 10000 open/close in 146.45 seconds: 68.28 ops/second - unlinked 0 (time 1763338169 ; total 0 ; last 0) total: 10000 unlinks in 81 seconds: 123.456787 unlinks/second - open/close 723 (time 1763338264.12 total 10.01 last 72.22) - open/close 1401 (time 1763338274.13 total 20.02 last 67.76) - open/close 2161 (time 1763338284.14 total 30.03 last 75.93) - open/close 2880 (time 1763338294.14 total 40.03 last 71.90) - open/close 3608 (time 1763338304.15 total 50.04 last 72.69) - open/close 4326 (time 1763338314.15 total 60.04 last 71.80) - open/close 5041 (time 1763338324.16 total 70.05 last 71.45) - open/close 5752 (time 1763338334.17 total 80.06 last 71.04) - open/close 6427 (time 1763338344.18 total 90.07 last 67.42) - open/close 7026 (time 1763338354.18 total 100.07 last 59.90) - open/close 7657 (time 1763338364.20 total 110.09 last 62.97) - open/close 8318 (time 1763338374.21 total 120.10 last 66.06) - open/close 8952 (time 1763338384.21 total 130.10 last 63.35) - open/close 9561 (time 1763338394.22 total 140.11 last 60.84) total: 10000 open/close in 146.64 seconds: 68.20 ops/second - unlinked 0 (time 1763338405 ; total 0 ; last 0) total: 10000 unlinks in 90 seconds: 111.111115 unlinks/second - open/close 632 (time 1763338510.10 total 10.01 last 63.16) - open/close 1267 (time 1763338520.11 total 20.02 last 63.43) - open/close 1976 (time 1763338530.12 total 30.03 last 70.80) - open/close 2673 (time 1763338540.13 total 40.04 last 69.66) - open/close 3340 (time 1763338550.14 total 50.05 last 66.64) - open/close 4055 (time 1763338560.14 total 60.05 last 71.45) - open/close 4750 (time 1763338570.15 total 70.06 last 69.46) - open/close 5437 (time 1763338580.16 total 80.07 last 68.63) - open/close 6184 (time 1763338590.16 total 90.07 last 74.70) - open/close 6813 (time 1763338600.16 total 100.07 last 62.90) - open/close 7444 (time 1763338610.17 total 110.08 last 63.06) - open/close 8148 (time 1763338620.17 total 120.09 last 70.34) - open/close 8816 (time 1763338630.18 total 130.09 last 66.74) - open/close 9516 (time 1763338640.19 total 140.10 last 69.94) total: 10000 open/close in 146.38 seconds: 68.32 ops/second - unlinked 0 (time 1763338650 ; total 0 ; last 0) total: 10000 unlinks in 89 seconds: 112.359550 unlinks/second - open/close 589 (time 1763338752.90 total 10.00 last 58.89) - open/close 1191 (time 1763338762.92 total 20.01 last 60.12) - open/close 1785 (time 1763338772.92 total 30.02 last 59.36) - open/close 2423 (time 1763338782.93 total 40.02 last 63.78) - open/close 3074 (time 1763338792.96 total 50.05 last 64.90) - open/close 3656 (time 1763338802.96 total 60.06 last 58.18) - open/close 4286 (time 1763338812.97 total 70.07 last 62.91) - open/close 4900 (time 1763338822.98 total 80.07 last 61.39) - open/close 5497 (time 1763338832.98 total 90.08 last 59.68) - open/close 6120 (time 1763338842.98 total 100.08 last 62.30) - open/close 6763 (time 1763338852.98 total 110.08 last 64.28) - open/close 7433 (time 1763338862.98 total 120.08 last 66.99) - open/close 8078 (time 1763338872.99 total 130.09 last 64.45) - open/close 8713 (time 1763338883.00 total 140.10 last 63.44) - open/close 9369 (time 1763338893.02 total 150.11 last 65.51) total: 10000 open/close in 159.90 seconds: 62.54 ops/second - unlinked 0 (time 1763338907 ; total 0 ; last 0) total: 10000 unlinks in 89 seconds: 112.359550 unlinks/second - open/close 727 (time 1763339010.91 total 10.00 last 72.69) - open/close 1465 (time 1763339020.91 total 20.01 last 73.76) - open/close 2191 (time 1763339030.92 total 30.02 last 72.51) - open/close 2900 (time 1763339040.93 total 40.03 last 70.84) - open/close 3639 (time 1763339050.94 total 50.04 last 73.84) - open/close 4366 (time 1763339060.96 total 60.05 last 72.58) - open/close 5095 (time 1763339070.96 total 70.06 last 72.88) - open/close 5856 (time 1763339080.97 total 80.07 last 76.02) - open/close 6667 (time 1763339090.97 total 90.07 last 81.08) - open/close 7411 (time 1763339100.98 total 100.08 last 74.34) - open/close 8175 (time 1763339110.98 total 110.08 last 76.39) - open/close 8863 (time 1763339120.99 total 120.09 last 68.72) - open/close 9610 (time 1763339131.01 total 130.10 last 74.59) total: 10000 open/close in 135.24 seconds: 73.94 ops/second - unlinked 0 (time 1763339139 ; total 0 ; last 0) total: 10000 unlinks in 81 seconds: 123.456787 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg447-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg447-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg447-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.204.147@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.204.147@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg447-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg447-server: oleg447-server.virtnet: executing set_default_debug -1 all pdsh@oleg447-client: oleg447-server: ssh exited with exit code 1 Started lustre-OST0000 oleg447-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg447-server: oleg447-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg447-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg447-client.virtnet: -o user_xattr,flock 192.168.204.147@tcp:/lustre /mnt/lustre On OST0, 14885 used inodes rc=0 pdsh@oleg447-client: oleg447-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg447-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg447-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg447-server Permanent disk data: Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: mgsnode=192.168.204.147@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.204.147@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=98 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre:OST0000 checking for existing Lustre data: found Read previous values: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: sys.timeout=20 autodegrade=on mgsnode=192.168.204.147@tcp Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: sys.timeout=20 autodegrade=on mgsnode=192.168.204.147@tcp autodegrade=on lustre:svname=lustre-OST0000 Writing lustre-ost1/ost1 properties lustre:sys.timeout=20 lustre:autodegrade=on lustre:mgsnode=192.168.204.147@tcp lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg447-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg447-server: oleg447-server.virtnet: executing set_default_debug -1 all pdsh@oleg447-client: oleg447-server: ssh exited with exit code 1 Started lustre-OST0000 oleg447-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg447-server: oleg447-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg447-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg447-client.virtnet: -o user_xattr,flock 192.168.204.147@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg447-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg447-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg447-server stop mds service on oleg447-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg447-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg447-server' oleg447-server: oleg447-server.virtnet: executing unload_modules_local oleg447-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg447-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg447-client: oleg447-client: ssh exited with exit code 2 pdsh@oleg447-client: oleg447-server: ssh exited with exit code 2 pdsh@oleg447-client: oleg447-client: ssh exited with exit code 2 pdsh@oleg447-client: oleg447-server: ssh exited with exit code 2 PASS 69 (2569s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 19:29:03 (1763339343) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (3s) Stopping clients: oleg447-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg447-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg447-client: oleg447-server: ssh exited with exit code 2 oleg447-server: oleg447-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg447-server' oleg447-server: oleg447-server.virtnet: executing load_modules_local oleg447-server: Loading modules from /home/green/git/lustre-release/lustre oleg447-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg447-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg447-server: detected 4 online CPUs by sysfs oleg447-server: MODOPTS_LIBCFS= oleg447-server: Force libcfs to create 2 CPU partitions oleg447-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg447-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 2960 sec ========== 19:30:00 (1763339400) === conf-sanity: start cleanup 19:30:02 (1763339402) === === conf-sanity: finish cleanup 19:30:05 (1763339405) ===