-----============= acceptance-small: conf-sanity ============----- Sun Nov 16 19:21:40 EST 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg218-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32b 32c 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg218-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg218-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg218-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg218-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg218-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg218-server oleg218-server: oleg218-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg218-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg218-server: oleg218-server.virtnet: executing set_default_debug -1 all pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg218-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg218-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg218-server: oleg218-server.virtnet: executing set_default_debug -1 all pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg218-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg218-server: oleg218-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg218-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg218-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg218-server stop mds service on oleg218-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg218-server == conf-sanity test 45: long unlink handling in ptlrpcd == 19:24:11 (1763339051) start mds service on oleg218-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg218-server: oleg218-server.virtnet: executing set_default_debug -1 all pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg218-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg218-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg218-server: oleg218-server.virtnet: executing set_default_debug -1 all pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1 Started lustre-OST0000 oleg218-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg218-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg218-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg218-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg218-server: oleg218-server.virtnet: executing set_default_debug -1 all pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg218-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg218-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg218-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg218-server stop mds service on oleg218-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg218-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing unload_modules_local oleg218-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg218-client: oleg218-client: ssh exited with exit code 2 pdsh@oleg218-client: oleg218-server: ssh exited with exit code 2 pdsh@oleg218-client: oleg218-server: ssh exited with exit code 2 pdsh@oleg218-client: oleg218-client: ssh exited with exit code 2 PASS 45 (202s) == conf-sanity test 69: replace an OST with the same index ========================================================== 19:27:33 (1763339253) start mds service on oleg218-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: Force libcfs to create 2 CPU partitions oleg218-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg218-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg218-server: oleg218-server.virtnet: executing set_default_debug -1 all pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg218-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg218-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg218-server: oleg218-server.virtnet: executing set_default_debug -1 all pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1 Started lustre-OST0000 oleg218-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 11872 inodes available. Want 99872. rc=0 - open/close 653 (time 1763339329.80 total 10.02 last 65.17) - open/close 1291 (time 1763339339.81 total 20.03 last 63.75) - open/close 1993 (time 1763339349.83 total 30.04 last 70.08) - open/close 2706 (time 1763339359.84 total 40.06 last 71.22) - open/close 3370 (time 1763339369.84 total 50.06 last 66.37) - open/close 4042 (time 1763339379.85 total 60.07 last 67.14) - open/close 4663 (time 1763339389.87 total 70.08 last 62.01) - open/close 5390 (time 1763339399.87 total 80.09 last 72.68) - open/close 6017 (time 1763339409.88 total 90.10 last 62.60) - open/close 6646 (time 1763339419.89 total 100.11 last 62.83) - open/close 7320 (time 1763339429.90 total 110.12 last 67.35) - open/close 8013 (time 1763339439.91 total 120.12 last 69.27) - open/close 8679 (time 1763339449.91 total 130.13 last 66.56) - open/close 9372 (time 1763339459.93 total 140.15 last 69.17) total: 10000 open/close in 149.32 seconds: 66.97 ops/second - unlinked 0 (time 1763339473 ; total 0 ; last 0) total: 10000 unlinks in 86 seconds: 116.279068 unlinks/second - open/close 695 (time 1763339573.30 total 10.00 last 69.50) - open/close 1400 (time 1763339583.31 total 20.01 last 70.41) - open/close 2120 (time 1763339593.32 total 30.02 last 71.93) - open/close 2799 (time 1763339603.32 total 40.03 last 67.88) - open/close 3468 (time 1763339613.34 total 50.04 last 66.80) - open/close 4119 (time 1763339623.34 total 60.05 last 65.05) - open/close 4763 (time 1763339633.35 total 70.06 last 64.34) - open/close 5407 (time 1763339643.36 total 80.06 last 64.36) - open/close 6086 (time 1763339653.37 total 90.08 last 67.81) - open/close 6789 (time 1763339663.38 total 100.09 last 70.23) - open/close 7498 (time 1763339673.39 total 110.09 last 70.84) - open/close 8168 (time 1763339683.39 total 120.10 last 66.98) - open/close 8817 (time 1763339693.40 total 130.10 last 64.88) - open/close 9479 (time 1763339703.41 total 140.11 last 66.13) total: 10000 open/close in 148.19 seconds: 67.48 ops/second - unlinked 0 (time 1763339716 ; total 0 ; last 0) total: 10000 unlinks in 88 seconds: 113.636360 unlinks/second - open/close 615 (time 1763339819.91 total 10.01 last 61.44) - open/close 1213 (time 1763339829.93 total 20.03 last 59.69) - open/close 1886 (time 1763339839.93 total 30.03 last 67.28) - open/close 2623 (time 1763339849.98 total 40.09 last 73.31) - open/close 3317 (time 1763339859.99 total 50.09 last 69.39) - open/close 3965 (time 1763339869.99 total 60.10 last 64.75) - open/close 4674 (time 1763339879.99 total 70.10 last 70.90) - open/close 5331 (time 1763339890.01 total 80.11 last 65.62) - open/close 5994 (time 1763339900.02 total 90.12 last 66.22) - open/close 6652 (time 1763339910.02 total 100.12 last 65.77) - open/close 7191 (time 1763339920.04 total 110.14 last 53.79) - open/close 7753 (time 1763339930.05 total 120.15 last 56.17) - open/close 8409 (time 1763339940.06 total 130.16 last 65.56) - open/close 9030 (time 1763339950.07 total 140.17 last 62.03) - open/close 9615 (time 1763339960.07 total 150.17 last 58.48) total: 10000 open/close in 157.18 seconds: 63.62 ops/second - unlinked 0 (time 1763339971 ; total 0 ; last 0) total: 10000 unlinks in 89 seconds: 112.359550 unlinks/second - open/close 784 (time 1763340073.98 total 10.00 last 78.39) - open/close 1533 (time 1763340083.98 total 20.00 last 74.88) - open/close 2243 (time 1763340093.98 total 30.01 last 70.99) - open/close 2922 (time 1763340103.99 total 40.01 last 67.86) - open/close 3649 (time 1763340113.99 total 50.01 last 72.70) - open/close 4307 (time 1763340123.99 total 60.01 last 65.79) - open/close 4977 (time 1763340134.00 total 70.02 last 66.96) - open/close 5646 (time 1763340144.00 total 80.02 last 66.88) - open/close 6283 (time 1763340154.00 total 90.03 last 63.67) - open/close 6948 (time 1763340164.01 total 100.03 last 66.47) - open/close 7627 (time 1763340174.01 total 110.03 last 67.88) - open/close 8335 (time 1763340184.02 total 120.04 last 70.75) - open/close 8970 (time 1763340194.03 total 130.06 last 63.41) - open/close 9627 (time 1763340204.05 total 140.07 last 65.59) total: 10000 open/close in 145.41 seconds: 68.77 ops/second - unlinked 0 (time 1763340213 ; total 0 ; last 0) total: 10000 unlinks in 85 seconds: 117.647057 unlinks/second - open/close 645 (time 1763340312.23 total 10.00 last 64.49) - open/close 1288 (time 1763340322.23 total 20.00 last 64.28) - open/close 1980 (time 1763340332.24 total 30.02 last 69.10) - open/close 2617 (time 1763340342.25 total 40.02 last 63.67) - open/close 3285 (time 1763340352.25 total 50.03 last 66.78) - open/close 3990 (time 1763340362.26 total 60.03 last 70.45) - open/close 4722 (time 1763340372.27 total 70.04 last 73.16) - open/close 5456 (time 1763340382.27 total 80.04 last 73.39) - open/close 6190 (time 1763340392.27 total 90.04 last 73.38) - open/close 6930 (time 1763340402.28 total 100.05 last 73.92) - open/close 7685 (time 1763340412.28 total 110.06 last 75.49) - open/close 8315 (time 1763340422.28 total 120.06 last 62.99) - open/close 8953 (time 1763340432.29 total 130.07 last 63.75) - open/close 9582 (time 1763340442.31 total 140.08 last 62.80) total: 10000 open/close in 147.00 seconds: 68.03 ops/second - unlinked 0 (time 1763340453 ; total 0 ; last 0) total: 10000 unlinks in 88 seconds: 113.636360 unlinks/second - open/close 736 (time 1763340555.89 total 10.01 last 73.52) - open/close 1382 (time 1763340565.90 total 20.02 last 64.56) - open/close 1975 (time 1763340575.90 total 30.02 last 59.25) - open/close 2640 (time 1763340585.91 total 40.03 last 66.45) - open/close 3349 (time 1763340595.92 total 50.04 last 70.84) - open/close 4062 (time 1763340605.93 total 60.06 last 71.19) - open/close 4813 (time 1763340615.95 total 70.07 last 75.02) - open/close 5545 (time 1763340625.96 total 80.08 last 73.13) - open/close 6276 (time 1763340635.96 total 90.08 last 73.09) - open/close 6999 (time 1763340645.96 total 100.08 last 72.30) - open/close 7696 (time 1763340655.98 total 110.10 last 69.57) - open/close 8386 (time 1763340665.99 total 120.11 last 68.93) - open/close 9093 (time 1763340675.99 total 130.11 last 70.66) - open/close 9761 (time 1763340685.99 total 140.11 last 66.79) total: 10000 open/close in 143.25 seconds: 69.81 ops/second - unlinked 0 (time 1763340692 ; total 0 ; last 0) total: 10000 unlinks in 86 seconds: 116.279068 unlinks/second - open/close 669 (time 1763340792.64 total 10.02 last 66.79) - open/close 1309 (time 1763340802.64 total 20.02 last 63.98) - open/close 1973 (time 1763340812.64 total 30.02 last 66.40) - open/close 2611 (time 1763340822.65 total 40.03 last 63.71) - open/close 3276 (time 1763340832.66 total 50.04 last 66.44) - open/close 4014 (time 1763340842.67 total 60.05 last 73.76) - open/close 4694 (time 1763340852.68 total 70.06 last 67.95) - open/close 5395 (time 1763340862.68 total 80.06 last 70.09) - open/close 6073 (time 1763340872.69 total 90.07 last 67.75) - open/close 6760 (time 1763340882.69 total 100.07 last 68.65) - open/close 7400 (time 1763340892.70 total 110.08 last 63.96) - open/close 8031 (time 1763340902.72 total 120.10 last 62.98) - open/close 8738 (time 1763340912.73 total 130.11 last 70.62) - open/close 9387 (time 1763340922.74 total 140.12 last 64.87) - open/close 9942 (time 1763340932.74 total 150.12 last 55.47) total: 10000 open/close in 151.05 seconds: 66.20 ops/second - unlinked 0 (time 1763340937 ; total 0 ; last 0) total: 10000 unlinks in 83 seconds: 120.481926 unlinks/second - open/close 609 (time 1763341034.14 total 10.01 last 60.83) - open/close 1265 (time 1763341044.14 total 20.02 last 65.56) - open/close 1917 (time 1763341054.15 total 30.03 last 65.15) - open/close 2551 (time 1763341064.17 total 40.04 last 63.29) - open/close 3210 (time 1763341074.18 total 50.06 last 65.79) - open/close 3898 (time 1763341084.20 total 60.08 last 68.66) - open/close 4530 (time 1763341094.20 total 70.08 last 63.20) - open/close 5216 (time 1763341104.21 total 80.09 last 68.53) - open/close 5868 (time 1763341114.22 total 90.09 last 65.16) - open/close 6515 (time 1763341124.23 total 100.11 last 64.61) - open/close 7190 (time 1763341134.24 total 110.12 last 67.44) - open/close 7859 (time 1763341144.26 total 120.13 last 66.82) - open/close 8464 (time 1763341154.27 total 130.14 last 60.41) - open/close 9190 (time 1763341164.27 total 140.15 last 72.58) - open/close 9895 (time 1763341174.28 total 150.16 last 70.43) total: 10000 open/close in 151.62 seconds: 65.95 ops/second - unlinked 0 (time 1763341178 ; total 0 ; last 0) total: 10000 unlinks in 86 seconds: 116.279068 unlinks/second - open/close 691 (time 1763341278.68 total 10.01 last 69.00) - open/close 1366 (time 1763341288.69 total 20.02 last 67.46) - open/close 2030 (time 1763341298.69 total 30.03 last 66.36) - open/close 2672 (time 1763341308.71 total 40.05 last 64.07) - open/close 3322 (time 1763341318.73 total 50.07 last 64.88) - open/close 3958 (time 1763341328.74 total 60.07 last 63.55) - open/close 4640 (time 1763341338.75 total 70.09 last 68.11) - open/close 5312 (time 1763341348.76 total 80.09 last 67.16) - open/close 5972 (time 1763341358.76 total 90.10 last 65.98) - open/close 6652 (time 1763341368.77 total 100.11 last 67.95) - open/close 7310 (time 1763341378.78 total 110.11 last 65.76) - open/close 7943 (time 1763341388.79 total 120.12 last 63.24) - open/close 8593 (time 1763341398.79 total 130.13 last 64.98) - open/close 9240 (time 1763341408.80 total 140.13 last 64.66) - open/close 9879 (time 1763341418.81 total 150.14 last 63.81) total: 10000 open/close in 151.74 seconds: 65.90 ops/second - unlinked 0 (time 1763341424 ; total 0 ; last 0) total: 10000 unlinks in 87 seconds: 114.942528 unlinks/second - open/close 723 (time 1763341525.66 total 10.00 last 72.27) - open/close 1494 (time 1763341535.68 total 20.02 last 76.95) - open/close 2159 (time 1763341545.69 total 30.03 last 66.47) - open/close 2866 (time 1763341555.70 total 40.04 last 70.63) - open/close 3635 (time 1763341565.71 total 50.05 last 76.84) - open/close 4339 (time 1763341575.71 total 60.05 last 70.38) - open/close 5046 (time 1763341585.72 total 70.06 last 70.62) - open/close 5725 (time 1763341595.75 total 80.09 last 67.69) - open/close 6343 (time 1763341605.76 total 90.10 last 61.77) - open/close 6969 (time 1763341615.77 total 100.11 last 62.52) - open/close 7588 (time 1763341625.78 total 110.12 last 61.85) - open/close 8222 (time 1763341635.78 total 120.12 last 63.40) - open/close 8954 (time 1763341645.78 total 130.12 last 73.16) - open/close 9645 (time 1763341655.81 total 140.15 last 68.92) total: 10000 open/close in 145.35 seconds: 68.80 ops/second - unlinked 0 (time 1763341664 ; total 0 ; last 0) total: 10000 unlinks in 86 seconds: 116.279068 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg218-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg218-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg218-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.202.118@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.202.118@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg218-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg218-server: oleg218-server.virtnet: executing set_default_debug -1 all pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1 Started lustre-OST0000 oleg218-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg218-server: oleg218-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg218-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre On OST0, 20325 used inodes rc=0 pdsh@oleg218-client: oleg218-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg218-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg218-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg218-server Permanent disk data: Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: mgsnode=192.168.202.118@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.202.118@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=98 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre:OST0000 checking for existing Lustre data: found Read previous values: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: mgsnode=192.168.202.118@tcp autodegrade=on sys.timeout=20 Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.202.118@tcp autodegrade=on sys.timeout=20 autodegrade=on lustre:svname=lustre-OST0000 Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.202.118@tcp lustre:autodegrade=on lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg218-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg218-server: oleg218-server.virtnet: executing set_default_debug -1 all pdsh@oleg218-client: oleg218-server: ssh exited with exit code 1 Started lustre-OST0000 oleg218-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg218-server: oleg218-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg218-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg218-client.virtnet: -o user_xattr,flock 192.168.202.118@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg218-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg218-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg218-server stop mds service on oleg218-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg218-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing unload_modules_local oleg218-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg218-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg218-client: oleg218-client: ssh exited with exit code 2 pdsh@oleg218-client: oleg218-server: ssh exited with exit code 2 pdsh@oleg218-client: oleg218-server: ssh exited with exit code 2 pdsh@oleg218-client: oleg218-client: ssh exited with exit code 2 PASS 69 (2612s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 20:11:06 (1763341866) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (4s) Stopping clients: oleg218-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg218-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg218-client: oleg218-server: ssh exited with exit code 2 oleg218-server: oleg218-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg218-server' oleg218-server: oleg218-server.virtnet: executing load_modules_local oleg218-server: Loading modules from /home/green/git/lustre-release/lustre oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1030: echo: write error: Device or resource busy oleg218-server: detected 4 online CPUs by sysfs oleg218-server: MODOPTS_LIBCFS= oleg218-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1031: echo: write error: Device or resource busy oleg218-server: Force libcfs to create 2 CPU partitions oleg218-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg218-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 3032 sec ========== 20:12:13 (1763341933) === conf-sanity: start cleanup 20:12:15 (1763341935) === === conf-sanity: finish cleanup 20:12:18 (1763341938) ===