-----============= acceptance-small: conf-sanity ============----- Mon Dec 22 09:21:44 EST 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg412-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32b 32c 32newtarball 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg412-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg412-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg412-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg412-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg412-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg412-server oleg412-server: oleg412-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg412-server' oleg412-server: oleg412-server.virtnet: executing load_modules_local oleg412-server: Loading modules from /home/green/git/lustre-release/lustre oleg412-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg412-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg412-server: detected 4 online CPUs by sysfs oleg412-server: MODOPTS_LIBCFS= oleg412-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg412-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg412-server: oleg412-server.virtnet: executing set_default_debug -1 all pdsh@oleg412-client: oleg412-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg412-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg412-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg412-server: oleg412-server.virtnet: executing set_default_debug -1 all pdsh@oleg412-client: oleg412-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg412-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg412-server: oleg412-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg412-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg412-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg412-server stop mds service on oleg412-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg412-server == conf-sanity test 45: long unlink handling in ptlrpcd == 09:24:12 (1766413452) start mds service on oleg412-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg412-server: oleg412-server.virtnet: executing set_default_debug -1 all pdsh@oleg412-client: oleg412-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg412-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg412-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg412-server: oleg412-server.virtnet: executing set_default_debug -1 all pdsh@oleg412-client: oleg412-server: ssh exited with exit code 1 Started lustre-OST0000 oleg412-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg412-client.virtnet: -o user_xattr,flock 192.168.204.112@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg412-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg412-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg412-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg412-server: oleg412-server.virtnet: executing set_default_debug -1 all pdsh@oleg412-client: oleg412-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg412-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg412-client.virtnet: -o user_xattr,flock 192.168.204.112@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg412-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg412-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg412-server stop mds service on oleg412-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg412-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg412-server' oleg412-server: oleg412-server.virtnet: executing unload_modules_local oleg412-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg412-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg412-client: oleg412-server: ssh exited with exit code 2 pdsh@oleg412-client: oleg412-client: ssh exited with exit code 2 pdsh@oleg412-client: oleg412-client: ssh exited with exit code 2 pdsh@oleg412-client: oleg412-server: ssh exited with exit code 2 PASS 45 (193s) == conf-sanity test 69: replace an OST with the same index ========================================================== 09:27:26 (1766413646) start mds service on oleg412-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg412-server' oleg412-server: oleg412-server.virtnet: executing load_modules_local oleg412-server: Loading modules from /home/green/git/lustre-release/lustre oleg412-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg412-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg412-server: detected 4 online CPUs by sysfs oleg412-server: MODOPTS_LIBCFS= oleg412-server: Force libcfs to create 2 CPU partitions oleg412-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg412-server: quota/lquota options: 'hash_lqs_cur_bits=3' Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg412-server: oleg412-server.virtnet: executing set_default_debug -1 all pdsh@oleg412-client: oleg412-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg412-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg412-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg412-server: oleg412-server.virtnet: executing set_default_debug -1 all pdsh@oleg412-client: oleg412-server: ssh exited with exit code 1 Started lustre-OST0000 oleg412-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg412-client.virtnet: -o user_xattr,flock 192.168.204.112@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 11872 inodes available. Want 99872. rc=0 - open/close 732 (time 1766413720.00 total 10.00 last 73.18) - open/close 1398 (time 1766413730.00 total 20.01 last 66.56) - open/close 2129 (time 1766413740.02 total 30.03 last 72.99) - open/close 2851 (time 1766413750.03 total 40.04 last 72.10) - open/close 3604 (time 1766413760.04 total 50.05 last 75.24) - open/close 4334 (time 1766413770.05 total 60.06 last 72.93) - open/close 5097 (time 1766413780.05 total 70.06 last 76.29) - open/close 5774 (time 1766413790.05 total 80.06 last 67.68) - open/close 6439 (time 1766413800.05 total 90.06 last 66.50) - open/close 7064 (time 1766413810.06 total 100.07 last 62.44) - open/close 7758 (time 1766413820.08 total 110.09 last 69.28) - open/close 8466 (time 1766413830.08 total 120.09 last 70.78) - open/close 9132 (time 1766413840.09 total 130.10 last 66.53) - open/close 9745 (time 1766413850.11 total 140.12 last 61.22) total: 10000 open/close in 144.29 seconds: 69.31 ops/second - unlinked 0 (time 1766413858 ; total 0 ; last 0) total: 10000 unlinks in 85 seconds: 117.647057 unlinks/second - open/close 621 (time 1766413956.94 total 10.01 last 62.05) - open/close 1230 (time 1766413966.96 total 20.02 last 60.82) - open/close 1805 (time 1766413976.96 total 30.02 last 57.50) - open/close 2450 (time 1766413986.96 total 40.03 last 64.47) - open/close 3185 (time 1766413996.96 total 50.03 last 73.49) - open/close 3906 (time 1766414006.97 total 60.03 last 72.07) - open/close 4602 (time 1766414016.98 total 70.05 last 69.51) - open/close 5291 (time 1766414026.98 total 80.05 last 68.88) - open/close 5916 (time 1766414037.01 total 90.07 last 62.34) - open/close 6541 (time 1766414047.02 total 100.09 last 62.42) - open/close 7141 (time 1766414057.02 total 110.09 last 60.00) - open/close 7784 (time 1766414067.03 total 120.10 last 64.24) - open/close 8455 (time 1766414077.05 total 130.11 last 67.00) - open/close 9076 (time 1766414087.07 total 140.13 last 61.96) - open/close 9697 (time 1766414097.08 total 150.14 last 62.04) total: 10000 open/close in 154.35 seconds: 64.79 ops/second - unlinked 0 (time 1766414105 ; total 0 ; last 0) total: 10000 unlinks in 89 seconds: 112.359550 unlinks/second - open/close 671 (time 1766414208.63 total 10.02 last 66.97) - open/close 1377 (time 1766414218.64 total 20.02 last 70.58) - open/close 2067 (time 1766414228.64 total 30.02 last 69.00) - open/close 2768 (time 1766414238.65 total 40.03 last 70.04) - open/close 3402 (time 1766414248.65 total 50.04 last 63.34) - open/close 4053 (time 1766414258.67 total 60.05 last 65.02) - open/close 4712 (time 1766414268.67 total 70.06 last 65.86) - open/close 5387 (time 1766414278.68 total 80.07 last 67.42) - open/close 6070 (time 1766414288.69 total 90.08 last 68.24) - open/close 6786 (time 1766414298.70 total 100.09 last 71.54) - open/close 7442 (time 1766414308.71 total 110.10 last 65.53) - open/close 8129 (time 1766414318.71 total 120.10 last 68.70) - open/close 8774 (time 1766414328.71 total 130.10 last 64.49) - open/close 9502 (time 1766414338.72 total 140.10 last 72.76) total: 10000 open/close in 147.19 seconds: 67.94 ops/second - unlinked 0 (time 1766414349 ; total 0 ; last 0) total: 10000 unlinks in 84 seconds: 119.047623 unlinks/second - open/close 774 (time 1766414446.15 total 10.00 last 77.38) - open/close 1537 (time 1766414456.16 total 20.02 last 76.19) - open/close 2335 (time 1766414466.17 total 30.02 last 79.75) - open/close 3084 (time 1766414476.18 total 40.03 last 74.86) - open/close 3829 (time 1766414486.19 total 50.04 last 74.42) - open/close 4552 (time 1766414496.19 total 60.04 last 72.30) - open/close 5248 (time 1766414506.19 total 70.04 last 69.59) - open/close 5938 (time 1766414516.19 total 80.04 last 68.98) - open/close 6626 (time 1766414526.19 total 90.05 last 68.79) - open/close 7290 (time 1766414536.19 total 100.05 last 66.39) - open/close 7918 (time 1766414546.21 total 110.06 last 62.71) - open/close 8587 (time 1766414556.22 total 120.07 last 66.80) - open/close 9231 (time 1766414566.22 total 130.08 last 64.39) - open/close 9873 (time 1766414576.23 total 140.08 last 64.16) total: 10000 open/close in 141.82 seconds: 70.51 ops/second - unlinked 0 (time 1766414581 ; total 0 ; last 0) total: 10000 unlinks in 88 seconds: 113.636360 unlinks/second - open/close 601 (time 1766414683.27 total 10.01 last 60.06) - open/close 1298 (time 1766414693.28 total 20.01 last 69.67) - open/close 1978 (time 1766414703.29 total 30.02 last 67.92) - open/close 2677 (time 1766414713.30 total 40.03 last 69.81) - open/close 3392 (time 1766414723.31 total 50.04 last 71.43) - open/close 4081 (time 1766414733.32 total 60.05 last 68.88) - open/close 4799 (time 1766414743.33 total 70.06 last 71.73) - open/close 5511 (time 1766414753.33 total 80.06 last 71.17) - open/close 6216 (time 1766414763.34 total 90.07 last 70.42) - open/close 6915 (time 1766414773.35 total 100.09 last 69.80) - open/close 7528 (time 1766414783.36 total 110.09 last 61.30) - open/close 8211 (time 1766414793.36 total 120.09 last 68.26) - open/close 8899 (time 1766414803.39 total 130.12 last 68.61) - open/close 9590 (time 1766414813.40 total 140.14 last 68.99) total: 10000 open/close in 146.18 seconds: 68.41 ops/second - unlinked 0 (time 1766414822 ; total 0 ; last 0) total: 10000 unlinks in 81 seconds: 123.456787 unlinks/second - open/close 760 (time 1766414917.31 total 10.02 last 75.88) - open/close 1452 (time 1766414927.32 total 20.03 last 69.12) - open/close 2037 (time 1766414937.32 total 30.03 last 58.49) - open/close 2647 (time 1766414947.34 total 40.04 last 60.92) - open/close 3427 (time 1766414957.34 total 50.04 last 77.99) - open/close 4130 (time 1766414967.34 total 60.04 last 70.30) - open/close 4865 (time 1766414977.34 total 70.04 last 73.50) - open/close 5618 (time 1766414987.36 total 80.06 last 75.17) - open/close 6389 (time 1766414997.37 total 90.07 last 77.00) - open/close 7134 (time 1766415007.38 total 100.09 last 74.39) - open/close 7864 (time 1766415017.39 total 110.10 last 72.95) - open/close 8611 (time 1766415027.40 total 120.10 last 74.64) - open/close 9313 (time 1766415037.40 total 130.11 last 70.16) total: 10000 open/close in 139.97 seconds: 71.45 ops/second - unlinked 0 (time 1766415051 ; total 0 ; last 0) total: 10000 unlinks in 80 seconds: 125.000000 unlinks/second - open/close 709 (time 1766415144.84 total 10.00 last 70.87) - open/close 1452 (time 1766415154.84 total 20.01 last 74.27) - open/close 2137 (time 1766415164.85 total 30.02 last 68.43) - open/close 2823 (time 1766415174.86 total 40.03 last 68.53) - open/close 3593 (time 1766415184.88 total 50.04 last 76.88) - open/close 4307 (time 1766415194.88 total 60.05 last 71.37) - open/close 5038 (time 1766415204.88 total 70.05 last 73.07) - open/close 5714 (time 1766415214.89 total 80.06 last 67.56) - open/close 6436 (time 1766415224.90 total 90.06 last 72.17) - open/close 7184 (time 1766415234.90 total 100.06 last 74.80) - open/close 7930 (time 1766415244.92 total 110.09 last 74.42) - open/close 8666 (time 1766415254.94 total 120.11 last 73.46) - open/close 9385 (time 1766415264.95 total 130.11 last 71.84) total: 10000 open/close in 138.12 seconds: 72.40 ops/second - unlinked 0 (time 1766415276 ; total 0 ; last 0) total: 10000 unlinks in 79 seconds: 126.582275 unlinks/second - open/close 753 (time 1766415369.75 total 10.01 last 75.23) - open/close 1491 (time 1766415379.76 total 20.01 last 73.78) - open/close 2278 (time 1766415389.77 total 30.02 last 78.62) - open/close 2995 (time 1766415399.78 total 40.04 last 71.59) - open/close 3753 (time 1766415409.80 total 50.05 last 75.68) - open/close 4505 (time 1766415419.80 total 60.06 last 75.19) - open/close 5292 (time 1766415429.81 total 70.07 last 78.62) - open/close 5996 (time 1766415439.82 total 80.08 last 70.31) - open/close 6728 (time 1766415449.84 total 90.10 last 73.07) - open/close 7396 (time 1766415459.85 total 100.11 last 66.74) - open/close 8054 (time 1766415469.85 total 110.11 last 65.76) - open/close 8789 (time 1766415479.87 total 120.13 last 73.36) - open/close 9403 (time 1766415489.88 total 130.14 last 61.36) total: 10000 open/close in 138.97 seconds: 71.96 ops/second - unlinked 0 (time 1766415502 ; total 0 ; last 0) total: 10000 unlinks in 75 seconds: 133.333328 unlinks/second - open/close 797 (time 1766415590.06 total 10.01 last 79.65) - open/close 1605 (time 1766415600.07 total 20.02 last 80.71) - open/close 2414 (time 1766415610.07 total 30.02 last 80.87) - open/close 3165 (time 1766415620.07 total 40.02 last 75.09) - open/close 3886 (time 1766415630.08 total 50.03 last 72.05) - open/close 4647 (time 1766415640.08 total 60.03 last 76.08) - open/close 5395 (time 1766415650.09 total 70.04 last 74.73) - open/close 6171 (time 1766415660.10 total 80.05 last 77.55) - open/close 6931 (time 1766415670.11 total 90.06 last 75.94) - open/close 7720 (time 1766415680.11 total 100.06 last 78.86) - open/close 8473 (time 1766415690.12 total 110.07 last 75.23) - open/close 9231 (time 1766415700.13 total 120.07 last 75.76) total: 10000 open/close in 129.49 seconds: 77.23 ops/second - unlinked 0 (time 1766415712 ; total 0 ; last 0) total: 10000 unlinks in 75 seconds: 133.333328 unlinks/second - open/close 706 (time 1766415801.33 total 10.01 last 70.56) - open/close 1394 (time 1766415811.33 total 20.01 last 68.79) - open/close 2131 (time 1766415821.34 total 30.01 last 73.66) - open/close 2835 (time 1766415831.34 total 40.02 last 70.36) - open/close 3555 (time 1766415841.35 total 50.02 last 71.96) - open/close 4272 (time 1766415851.35 total 60.03 last 71.69) - open/close 5003 (time 1766415861.37 total 70.05 last 72.95) - open/close 5692 (time 1766415871.38 total 80.06 last 68.84) - open/close 6463 (time 1766415881.40 total 90.07 last 76.96) - open/close 7195 (time 1766415891.40 total 100.08 last 73.18) - open/close 7903 (time 1766415901.41 total 110.08 last 70.76) - open/close 8700 (time 1766415911.42 total 120.09 last 79.64) - open/close 9465 (time 1766415921.42 total 130.10 last 76.44) total: 10000 open/close in 137.99 seconds: 72.47 ops/second - unlinked 0 (time 1766415932 ; total 0 ; last 0) total: 10000 unlinks in 80 seconds: 125.000000 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg412-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg412-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg412-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.204.112@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.204.112@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg412-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg412-server: oleg412-server.virtnet: executing set_default_debug -1 all pdsh@oleg412-client: oleg412-server: ssh exited with exit code 1 Started lustre-OST0000 oleg412-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg412-server: oleg412-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg412-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec mount lustre on /mnt/lustre..... Starting client: oleg412-client.virtnet: -o user_xattr,flock 192.168.204.112@tcp:/lustre /mnt/lustre On OST0, 7477 used inodes rc=0 pdsh@oleg412-client: oleg412-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg412-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg412-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg412-server Permanent disk data: Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: mgsnode=192.168.204.112@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.204.112@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=98 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre:OST0000 checking for existing Lustre data: found Read previous values: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: sys.timeout=20 autodegrade=on mgsnode=192.168.204.112@tcp Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: sys.timeout=20 autodegrade=on mgsnode=192.168.204.112@tcp autodegrade=on lustre:svname=lustre-OST0000 Writing lustre-ost1/ost1 properties lustre:sys.timeout=20 lustre:autodegrade=on lustre:mgsnode=192.168.204.112@tcp lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg412-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg412-server: oleg412-server.virtnet: executing set_default_debug -1 all pdsh@oleg412-client: oleg412-server: ssh exited with exit code 1 Started lustre-OST0000 oleg412-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg412-server: oleg412-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg412-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg412-client.virtnet: -o user_xattr,flock 192.168.204.112@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg412-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg412-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg412-server stop mds service on oleg412-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg412-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy unloading modules via unload_modules_local on: 'oleg412-server' oleg412-server: oleg412-server.virtnet: executing unload_modules_local oleg412-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy oleg412-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy modules unloaded. pdsh@oleg412-client: oleg412-client: ssh exited with exit code 2 pdsh@oleg412-client: oleg412-server: ssh exited with exit code 2 pdsh@oleg412-client: oleg412-client: ssh exited with exit code 2 pdsh@oleg412-client: oleg412-server: ssh exited with exit code 2 PASS 69 (2471s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 10:08:38 (1766416118) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (3s) Stopping clients: oleg412-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg412-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg412-client: oleg412-server: ssh exited with exit code 2 oleg412-server: oleg412-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' ln: failed to create symbolic link '/sbin/.libs': Read-only file system loading modules on: 'oleg412-server' oleg412-server: oleg412-server.virtnet: executing load_modules_local oleg412-server: Loading modules from /home/green/git/lustre-release/lustre oleg412-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1033: echo: write error: Device or resource busy oleg412-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg412-server: detected 4 online CPUs by sysfs oleg412-server: MODOPTS_LIBCFS= oleg412-server: Force libcfs to create 2 CPU partitions oleg412-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg412-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 2859 sec ========== 10:09:24 (1766416164) === conf-sanity: start cleanup 10:09:26 (1766416166) === === conf-sanity: finish cleanup 10:09:29 (1766416169) ===