-----============= acceptance-small: conf-sanity ============----- Wed Sep 16 21:15:35 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg340-server: /home/green/git/lustre-release/lustre/tests/except/conf-sanity.ex - see CLIENT_VERSION >= v2_15_63-53-g1a6ef725c2 (34683431 >= 34553653) for LU-16938, go 81 - see CLIENT_VERSION >= v2_15_65-31-g74a5b9efaa (34683431 >= 34554143) for LU-18348, go 32b 32c - see CLIENT_VERSION >= v2_16_55-19-gf69f27cc35 (34683431 >= 34617107) for LU-17920, go 123ae 123ag - see CLIENT_VERSION >= v2_17_52-94-g86904ef943 (34683431 >= 34681950) for LU-19740, go 115 excepting tests: 32b 32c 32newtarball 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg340-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg340-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg340-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg340-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg340-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg340-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg340-server oleg340-server: oleg340-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: kmemleak disabled oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: MODOPTS_LIBCFS= oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format mds2: lustre-mdt2/mdt2 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg340-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 start mds service on oleg340-server Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Commit the device label on lustre-mdt2/mdt2 Started lustre-MDT0001 oleg340-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg340-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg340-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg340-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg340-server: oleg340-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg340-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg340-server: oleg340-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg340-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg340-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg340-server stop mds service on oleg340-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg340-server stop mds service on oleg340-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg340-server == conf-sanity test 45: long unlink handling in ptlrpcd == 21:19:22 (1789607962) start mds service on oleg340-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg340-server Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg340-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg340-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg340-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Started lustre-OST0000 oleg340-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg340-client.virtnet: -o user_xattr,flock 192.168.203.140@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg340-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg340-server stop mds service on oleg340-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg340-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg340-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg340-server Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg340-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg340-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg340-client.virtnet: -o user_xattr,flock 192.168.203.140@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg340-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg340-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg340-server stop mds service on oleg340-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg340-server stop mds service on oleg340-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg340-server unloading modules via unload_modules_local on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing unload_modules_local oleg340-server: kmemleak disabled modules unloaded. pdsh@oleg340-client: oleg340-server: ssh exited with exit code 2 pdsh@oleg340-client: oleg340-client: ssh exited with exit code 2 pdsh@oleg340-client: oleg340-server: ssh exited with exit code 2 pdsh@oleg340-client: oleg340-client: ssh exited with exit code 2 PASS 45 (286s) == conf-sanity test 69: replace an OST with the same index ========================================================== 21:24:09 (1789608249) start mds service on oleg340-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' -rwxr-xr-x. 1 green green 1931640 Sep 16 13:21 /home/green/git/lustre-release/lustre/utils/mount.lustre -rwxr-xr-x. 1 root root 12 Jan 31 2019 /usr/sbin/mount.lustre mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: kmemleak disabled oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: MODOPTS_LIBCFS= oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg340-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg340-server: mdt/mdt options: 'mdt_enable_flr_ec=1' oleg340-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg340-server Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg340-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg340-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg340-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Started lustre-OST0000 oleg340-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg340-client.virtnet: -o user_xattr,flock 192.168.203.140@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 11904 inodes available. Want 99872. rc=0 - open/close 713 (time 1789608367.37 total 10.01 last 71.26) - open/close 1408 (time 1789608377.37 total 20.01 last 69.50) - open/close 2144 (time 1789608387.37 total 30.01 last 73.58) - open/close 2834 (time 1789608397.37 total 40.01 last 69.00) - open/close 3599 (time 1789608407.38 total 50.01 last 76.45) - open/close 4349 (time 1789608417.40 total 60.03 last 74.85) - open/close 5112 (time 1789608427.41 total 70.04 last 76.27) - open/close 5796 (time 1789608437.41 total 80.04 last 68.39) - open/close 6540 (time 1789608447.42 total 90.05 last 74.34) - open/close 7177 (time 1789608457.43 total 100.06 last 63.63) - open/close 7904 (time 1789608467.44 total 110.07 last 72.62) - open/close 8644 (time 1789608477.46 total 120.09 last 73.86) - open/close 9449 (time 1789608487.46 total 130.09 last 80.46) total: 10000 open/close in 138.07 seconds: 72.42 ops/second - unlinked 0 (time 1789608500 ; total 0 ; last 0) total: 10000 unlinks in 90 seconds: 111.111115 unlinks/second - open/close 812 (time 1789608606.09 total 10.00 last 81.17) - open/close 1526 (time 1789608616.10 total 20.01 last 71.36) - open/close 2275 (time 1789608626.11 total 30.02 last 74.78) - open/close 3054 (time 1789608636.12 total 40.03 last 77.84) - open/close 3791 (time 1789608646.12 total 50.04 last 73.67) - open/close 4415 (time 1789608656.14 total 60.05 last 62.30) - open/close 5081 (time 1789608666.14 total 70.05 last 66.58) - open/close 5746 (time 1789608676.15 total 80.06 last 66.44) - open/close 6320 (time 1789608686.18 total 90.09 last 57.26) - open/close 6978 (time 1789608696.20 total 100.11 last 65.65) - open/close 7640 (time 1789608706.20 total 110.11 last 66.20) - open/close 8318 (time 1789608716.20 total 120.11 last 67.79) - open/close 9028 (time 1789608726.21 total 130.12 last 70.97) - open/close 9756 (time 1789608736.21 total 140.12 last 72.78) total: 10000 open/close in 143.58 seconds: 69.65 ops/second - unlinked 0 (time 1789608744 ; total 0 ; last 0) total: 10000 unlinks in 86 seconds: 116.279068 unlinks/second - open/close 707 (time 1789608844.99 total 10.01 last 70.66) - open/close 1358 (time 1789608855.00 total 20.01 last 65.05) - open/close 1916 (time 1789608865.03 total 30.04 last 55.65) - open/close 2617 (time 1789608875.03 total 40.04 last 70.10) - open/close 3280 (time 1789608885.03 total 50.05 last 66.25) - open/close 4060 (time 1789608895.04 total 60.06 last 77.96) - open/close 4867 (time 1789608905.04 total 70.06 last 80.70) - open/close 5552 (time 1789608915.06 total 80.07 last 68.38) - open/close 6318 (time 1789608925.07 total 90.08 last 76.54) - open/close 7014 (time 1789608935.07 total 100.09 last 69.57) - open/close 7555 (time 1789608945.07 total 110.09 last 54.09) - open/close 8212 (time 1789608955.08 total 120.09 last 65.66) - open/close 8852 (time 1789608965.09 total 130.10 last 63.95) - open/close 9406 (time 1789608975.09 total 140.11 last 55.36) - open/close 9869 (time 1789608985.10 total 150.12 last 46.27) total: 10000 open/close in 151.92 seconds: 65.82 ops/second - unlinked 0 (time 1789608991 ; total 1 ; last 1) total: 10000 unlinks in 84 seconds: 119.047623 unlinks/second - open/close 749 (time 1789609089.34 total 10.01 last 74.83) - open/close 1396 (time 1789609099.37 total 20.04 last 64.53) - open/close 2126 (time 1789609109.38 total 30.05 last 72.93) - open/close 2840 (time 1789609119.39 total 40.06 last 71.27) - open/close 3590 (time 1789609129.40 total 50.07 last 74.95) - open/close 4339 (time 1789609139.41 total 60.08 last 74.81) - open/close 4993 (time 1789609149.42 total 70.09 last 65.36) - open/close 5680 (time 1789609159.42 total 80.09 last 68.67) - open/close 6363 (time 1789609169.46 total 90.13 last 68.01) - open/close 7039 (time 1789609179.46 total 100.14 last 67.60) - open/close 7688 (time 1789609189.47 total 110.14 last 64.85) - open/close 8331 (time 1789609199.48 total 120.15 last 64.22) - open/close 9021 (time 1789609209.48 total 130.16 last 69.00) - open/close 9728 (time 1789609219.49 total 140.17 last 70.63) total: 10000 open/close in 143.88 seconds: 69.50 ops/second - unlinked 0 (time 1789609227 ; total 0 ; last 0) total: 10000 unlinks in 89 seconds: 112.359550 unlinks/second - open/close 748 (time 1789609331.09 total 10.00 last 74.79) - open/close 1301 (time 1789609341.11 total 20.02 last 55.19) - open/close 1970 (time 1789609351.11 total 30.02 last 66.90) - open/close 2597 (time 1789609361.11 total 40.03 last 62.67) - open/close 3228 (time 1789609371.13 total 50.05 last 62.97) - open/close 3835 (time 1789609381.15 total 60.06 last 60.63) - open/close 4368 (time 1789609391.16 total 70.07 last 53.24) - open/close 5041 (time 1789609401.16 total 80.07 last 67.29) - open/close 5709 (time 1789609411.16 total 90.07 last 66.79) - open/close 6407 (time 1789609421.16 total 100.08 last 69.78) - open/close 7112 (time 1789609431.19 total 110.10 last 70.30) - open/close 7725 (time 1789609441.24 total 120.15 last 61.00) - open/close 8311 (time 1789609451.25 total 130.16 last 58.57) - open/close 8969 (time 1789609461.25 total 140.16 last 65.76) - open/close 9643 (time 1789609471.27 total 150.19 last 67.25) total: 10000 open/close in 156.00 seconds: 64.10 ops/second - unlinked 0 (time 1789609481 ; total 0 ; last 0) total: 10000 unlinks in 91 seconds: 109.890106 unlinks/second - open/close 624 (time 1789609586.69 total 10.01 last 62.33) - open/close 1131 (time 1789609596.69 total 20.01 last 50.70) - open/close 1794 (time 1789609606.70 total 30.03 last 66.20) - open/close 2486 (time 1789609616.71 total 40.03 last 69.18) - open/close 3131 (time 1789609626.72 total 50.04 last 64.43) - open/close 3831 (time 1789609636.73 total 60.05 last 69.95) - open/close 4469 (time 1789609646.75 total 70.08 last 63.62) - open/close 5143 (time 1789609656.76 total 80.08 last 67.38) - open/close 5826 (time 1789609666.79 total 90.11 last 68.10) - open/close 6445 (time 1789609676.79 total 100.11 last 61.88) - open/close 7036 (time 1789609686.79 total 110.12 last 59.07) - open/close 7760 (time 1789609696.81 total 120.14 last 72.28) - open/close 8452 (time 1789609706.81 total 130.14 last 69.20) - open/close 9090 (time 1789609716.81 total 140.14 last 63.80) - open/close 9779 (time 1789609726.82 total 150.15 last 68.83) total: 10000 open/close in 153.16 seconds: 65.29 ops/second - unlinked 0 (time 1789609733 ; total 0 ; last 0) total: 10000 unlinks in 90 seconds: 111.111115 unlinks/second - open/close 583 (time 1789609838.04 total 10.01 last 58.26) - open/close 1158 (time 1789609848.05 total 20.01 last 57.48) - open/close 1741 (time 1789609858.05 total 30.02 last 58.27) - open/close 2372 (time 1789609868.06 total 40.02 last 63.06) - open/close 3034 (time 1789609878.07 total 50.03 last 66.11) - open/close 3648 (time 1789609888.09 total 60.05 last 61.31) - open/close 4312 (time 1789609898.09 total 70.05 last 66.38) - open/close 4895 (time 1789609908.10 total 80.06 last 58.26) - open/close 5483 (time 1789609918.10 total 90.06 last 58.80) - open/close 6049 (time 1789609928.10 total 100.07 last 56.55) - open/close 6748 (time 1789609938.12 total 110.08 last 69.79) - open/close 7352 (time 1789609948.12 total 120.08 last 60.40) - open/close 8087 (time 1789609958.12 total 130.08 last 73.49) - open/close 8719 (time 1789609968.13 total 140.09 last 63.18) - open/close 9323 (time 1789609978.13 total 150.09 last 60.37) - open/close 9966 (time 1789609988.13 total 160.09 last 64.29) total: 10000 open/close in 160.71 seconds: 62.22 ops/second - unlinked 0 (time 1789609993 ; total 0 ; last 0) total: 10000 unlinks in 96 seconds: 104.166664 unlinks/second - open/close 773 (time 1789610104.52 total 10.01 last 77.21) - open/close 1451 (time 1789610114.53 total 20.02 last 67.73) - open/close 2175 (time 1789610124.54 total 30.03 last 72.34) - open/close 2810 (time 1789610134.55 total 40.04 last 63.45) - open/close 3404 (time 1789610144.57 total 50.06 last 59.28) - open/close 4003 (time 1789610154.57 total 60.06 last 59.89) - open/close 4524 (time 1789610164.58 total 70.07 last 52.06) - open/close 5172 (time 1789610174.61 total 80.10 last 64.61) - open/close 5944 (time 1789610184.62 total 90.11 last 77.12) - open/close 6620 (time 1789610194.63 total 100.12 last 67.53) - open/close 7336 (time 1789610204.64 total 110.13 last 71.55) - open/close 8036 (time 1789610214.64 total 120.13 last 69.95) - open/close 8631 (time 1789610224.67 total 130.16 last 59.34) - open/close 9322 (time 1789610234.67 total 140.16 last 69.08) total: 10000 open/close in 150.06 seconds: 66.64 ops/second - unlinked 0 (time 1789610249 ; total 0 ; last 0) total: 10000 unlinks in 88 seconds: 113.636360 unlinks/second - open/close 678 (time 1789610352.21 total 10.00 last 67.80) - open/close 1367 (time 1789610362.23 total 20.01 last 68.81) - open/close 2089 (time 1789610372.23 total 30.01 last 72.19) - open/close 2826 (time 1789610382.24 total 40.02 last 73.64) - open/close 3583 (time 1789610392.24 total 50.02 last 75.69) - open/close 4315 (time 1789610402.24 total 60.03 last 73.17) - open/close 5027 (time 1789610412.24 total 70.03 last 71.19) - open/close 5712 (time 1789610422.25 total 80.04 last 68.45) - open/close 6385 (time 1789610432.25 total 90.04 last 67.28) - open/close 7123 (time 1789610442.27 total 100.05 last 73.71) - open/close 7874 (time 1789610452.28 total 110.06 last 75.02) - open/close 8553 (time 1789610462.30 total 120.08 last 67.78) - open/close 9177 (time 1789610472.31 total 130.10 last 62.29) - open/close 9836 (time 1789610482.32 total 140.10 last 65.86) total: 10000 open/close in 142.95 seconds: 69.96 ops/second - unlinked 0 (time 1789610489 ; total 0 ; last 0) total: 10000 unlinks in 92 seconds: 108.695656 unlinks/second - open/close 722 (time 1789610596.14 total 10.02 last 72.04) - open/close 1430 (time 1789610606.16 total 20.04 last 70.67) - open/close 2198 (time 1789610616.18 total 30.06 last 76.67) - open/close 2935 (time 1789610626.18 total 40.07 last 73.64) - open/close 3706 (time 1789610636.19 total 50.07 last 77.03) - open/close 4415 (time 1789610646.20 total 60.08 last 70.88) - open/close 5177 (time 1789610656.20 total 70.08 last 76.16) - open/close 5905 (time 1789610666.22 total 80.10 last 72.69) - open/close 6607 (time 1789610676.22 total 90.10 last 70.17) - open/close 7312 (time 1789610686.22 total 100.10 last 70.48) - open/close 8063 (time 1789610696.23 total 110.11 last 75.06) - open/close 8793 (time 1789610706.24 total 120.12 last 72.92) - open/close 9487 (time 1789610716.25 total 130.13 last 69.32) total: 10000 open/close in 137.96 seconds: 72.48 ops/second - unlinked 0 (time 1789610728 ; total 0 ; last 0) total: 10000 unlinks in 88 seconds: 113.636360 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg340-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg340-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg340-server Permanent disk data: Device: lustre-ost1/ost1 Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.203.140@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.203.140@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg340-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Started lustre-OST0000 oleg340-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg340-server: oleg340-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg340-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg340-server: oleg340-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg340-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg340-client.virtnet: -o user_xattr,flock 192.168.203.140@tcp:/lustre /mnt/lustre On OST0, 20520 used inodes rc=0 pdsh@oleg340-client: oleg340-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg340-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg340-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg340-server Permanent disk data: Device: lustre-ost1/ost1 Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: mgsnode=192.168.203.140@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.203.140@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=98 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre:OST0000 checking for existing Lustre data: found Read previous values: Device: lustre-ost1/ost1 Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: mgsnode=192.168.203.140@tcp sys.timeout=20 autodegrade=on Permanent disk data: Device: lustre-ost1/ost1 Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.203.140@tcp sys.timeout=20 autodegrade=on autodegrade=on lustre:svname=lustre-OST0000 Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.203.140@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg340-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg340-server: oleg340-server.virtnet: executing set_default_debug -1 all pdsh@oleg340-client: oleg340-server: ssh exited with exit code 1 Started lustre-OST0000 oleg340-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg340-server: oleg340-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg340-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg340-server: oleg340-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg340-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg340-client.virtnet: -o user_xattr,flock 192.168.203.140@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg340-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg340-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg340-server stop mds service on oleg340-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg340-server stop mds service on oleg340-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg340-server unloading modules via unload_modules_local on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing unload_modules_local oleg340-server: kmemleak disabled modules unloaded. pdsh@oleg340-client: oleg340-client: ssh exited with exit code 2 pdsh@oleg340-client: oleg340-server: ssh exited with exit code 2 pdsh@oleg340-client: oleg340-server: ssh exited with exit code 2 pdsh@oleg340-client: oleg340-client: ssh exited with exit code 2 PASS 69 (2737s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 22:09:47 (1789610987) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (3s) Stopping clients: oleg340-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg340-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg340-client: oleg340-server: ssh exited with exit code 2 oleg340-server: oleg340-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' -rwxr-xr-x. 1 green green 1931640 Sep 16 13:21 /home/green/git/lustre-release/lustre/utils/mount.lustre -rwxr-xr-x. 1 root root 12 Jan 31 2019 /usr/sbin/mount.lustre mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre loading modules on: 'oleg340-server' oleg340-server: oleg340-server.virtnet: executing load_modules_local oleg340-server: kmemleak disabled oleg340-server: Loading modules from /home/green/git/lustre-release/lustre oleg340-server: detected 4 online CPUs by sysfs oleg340-server: MODOPTS_LIBCFS= oleg340-server: Force libcfs to create 2 CPU partitions oleg340-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg340-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg340-server: mdt/mdt options: 'mdt_enable_flr_ec=1' oleg340-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format mds2: lustre-mdt2/mdt2 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 3348 sec ========== 22:11:25 (1789611085) === conf-sanity: start cleanup 22:11:27 (1789611087) === === conf-sanity: finish cleanup 22:11:30 (1789611090) ===