-----============= acceptance-small: conf-sanity ============----- Fri Oct 2 01:29:09 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky use CONF_SANITY_ONLY='' CONF_SANITY_EXCEPT='' oleg609-server: /home/green/git/lustre-release/lustre/tests/except/conf-sanity.ex - see CLIENT_VERSION >= v2_15_63-53-g1a6ef725c2 (34683650 >= 34553653) for LU-16938, go 81 - see CLIENT_VERSION >= v2_15_65-31-g74a5b9efaa (34683650 >= 34554143) for LU-18348, go 32b 32c - see CLIENT_VERSION >= v2_16_55-19-gf69f27cc35 (34683650 >= 34617107) for LU-17920, go 123ae 123ag - see CLIENT_VERSION >= v2_17_52-94-g86904ef943 (34683650 >= 34681950) for LU-19740, go 115 only running tests: 45 69 111 excepting tests: 32b 32c 32newtarball 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg609-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg609-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg609-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg609-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg609-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg609-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg609-server oleg609-server: oleg609-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg609-server' oleg609-server: oleg609-server.virtnet: executing load_modules_local oleg609-server: kmemleak disabled oleg609-server: Loading modules from /home/green/git/lustre-release/lustre oleg609-server: detected 4 online CPUs by sysfs oleg609-server: MODOPTS_LIBCFS= oleg609-server: Force libcfs to create 2 CPU partitions oleg609-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format mds2: lustre-mdt2/mdt2 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg609-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg609-server: oleg609-server.virtnet: executing set_default_debug -1 all pdsh@oleg609-client: oleg609-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 start mds service on oleg609-server Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg609-server: oleg609-server.virtnet: executing set_default_debug -1 all pdsh@oleg609-client: oleg609-server: ssh exited with exit code 1 Commit the device label on lustre-mdt2/mdt2 Started lustre-MDT0001 oleg609-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg609-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg609-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg609-server: oleg609-server.virtnet: executing set_default_debug -1 all pdsh@oleg609-client: oleg609-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg609-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg609-server: oleg609-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg609-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg609-server: oleg609-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg609-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg609-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg609-server stop mds service on oleg609-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg609-server stop mds service on oleg609-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg609-server repeat= (ONLY_REPEAT=) repeat_end= (ONLY_MINUTES=) == conf-sanity test 45: long unlink handling in ptlrpcd == 01:32:32 (1790919152) start mds service on oleg609-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg609-server: oleg609-server.virtnet: executing set_default_debug -1 all pdsh@oleg609-client: oleg609-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg609-server Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg609-server: oleg609-server.virtnet: executing set_default_debug -1 all pdsh@oleg609-client: oleg609-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg609-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg609-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg609-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg609-server: oleg609-server.virtnet: executing set_default_debug -1 all pdsh@oleg609-client: oleg609-server: ssh exited with exit code 1 Started lustre-OST0000 oleg609-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg609-client.virtnet: -o user_xattr,flock 192.168.206.109@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg609-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg609-server stop mds service on oleg609-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg609-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg609-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg609-server: oleg609-server.virtnet: executing set_default_debug -1 all pdsh@oleg609-client: oleg609-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg609-server Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg609-server: oleg609-server.virtnet: executing set_default_debug -1 all pdsh@oleg609-client: oleg609-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg609-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg609-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg609-client.virtnet: -o user_xattr,flock 192.168.206.109@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg609-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg609-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg609-server stop mds service on oleg609-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg609-server stop mds service on oleg609-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg609-server unloading modules via unload_modules_local on: 'oleg609-server' oleg609-server: oleg609-server.virtnet: executing unload_modules_local oleg609-server: kmemleak disabled modules unloaded. pdsh@oleg609-client: oleg609-client: ssh exited with exit code 2 pdsh@oleg609-client: oleg609-server: ssh exited with exit code 2 pdsh@oleg609-client: oleg609-client: ssh exited with exit code 2 pdsh@oleg609-client: oleg609-server: ssh exited with exit code 2 PASS 45 (273s) repeat= (ONLY_REPEAT=) repeat_end= (ONLY_MINUTES=) == conf-sanity test 69: replace an OST with the same index ========================================================== 01:37:06 (1790919426) start mds service on oleg609-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' -rwxr-xr-x. 1 green green 1946968 Oct 1 19:15 /home/green/git/lustre-release/lustre/utils/mount.lustre -rwxr-xr-x. 1 root root 12 Jan 31 2019 /usr/sbin/mount.lustre mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre loading modules on: 'oleg609-server' oleg609-server: oleg609-server.virtnet: executing load_modules_local oleg609-server: kmemleak disabled oleg609-server: Loading modules from /home/green/git/lustre-release/lustre oleg609-server: detected 4 online CPUs by sysfs oleg609-server: MODOPTS_LIBCFS= oleg609-server: Force libcfs to create 2 CPU partitions oleg609-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg609-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg609-server: mdt/mdt options: 'mdt_enable_flr_ec=1' oleg609-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg609-server: oleg609-server.virtnet: executing set_default_debug -1 all pdsh@oleg609-client: oleg609-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg609-server Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg609-server: oleg609-server.virtnet: executing set_default_debug -1 all pdsh@oleg609-client: oleg609-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg609-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg609-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg609-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg609-server: oleg609-server.virtnet: executing set_default_debug -1 all pdsh@oleg609-client: oleg609-server: ssh exited with exit code 1 Started lustre-OST0000 oleg609-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg609-client.virtnet: -o user_xattr,flock 192.168.206.109@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 11904 inodes available. Want 99872. rc=0 - open/close 697 (time 1790919537.31 total 10.00 last 69.68) - open/close 1449 (time 1790919547.32 total 20.01 last 75.13) - open/close 2175 (time 1790919557.33 total 30.03 last 72.50) - open/close 2897 (time 1790919567.34 total 40.03 last 72.17) - open/close 3600 (time 1790919577.34 total 50.03 last 70.30) - open/close 4364 (time 1790919587.34 total 60.03 last 76.36) - open/close 5167 (time 1790919597.36 total 70.05 last 80.17) - open/close 5963 (time 1790919607.36 total 80.06 last 79.55) - open/close 6721 (time 1790919617.38 total 90.07 last 75.71) - open/close 7443 (time 1790919627.39 total 100.08 last 72.12) - open/close 8161 (time 1790919637.40 total 110.10 last 71.68) - open/close 8882 (time 1790919647.42 total 120.11 last 71.99) - open/close 9562 (time 1790919657.44 total 130.14 last 67.85) total: 10000 open/close in 136.29 seconds: 73.37 ops/second - unlinked 0 (time 1790919668 ; total 0 ; last 0) total: 10000 unlinks in 87 seconds: 114.942528 unlinks/second - open/close 780 (time 1790919771.27 total 10.02 last 77.88) - open/close 1539 (time 1790919781.28 total 20.02 last 75.85) - open/close 2252 (time 1790919791.29 total 30.03 last 71.25) - open/close 2987 (time 1790919801.30 total 40.04 last 73.44) - open/close 3727 (time 1790919811.30 total 50.04 last 73.98) - open/close 4391 (time 1790919821.30 total 60.04 last 66.39) - open/close 5119 (time 1790919831.31 total 70.05 last 72.76) - open/close 5880 (time 1790919841.31 total 80.05 last 76.05) - open/close 6585 (time 1790919851.32 total 90.06 last 70.48) - open/close 7290 (time 1790919861.32 total 100.06 last 70.46) - open/close 7943 (time 1790919871.33 total 110.07 last 65.24) - open/close 8605 (time 1790919881.34 total 120.08 last 66.15) - open/close 9256 (time 1790919891.35 total 130.09 last 65.04) total: 10000 open/close in 139.85 seconds: 71.51 ops/second - unlinked 0 (time 1790919905 ; total 0 ; last 0) total: 10000 unlinks in 89 seconds: 112.359550 unlinks/second - open/close 758 (time 1790920009.27 total 10.00 last 75.79) - open/close 1463 (time 1790920019.28 total 20.01 last 70.43) - open/close 2142 (time 1790920029.28 total 30.01 last 67.88) - open/close 2851 (time 1790920039.30 total 40.03 last 70.76) - open/close 3461 (time 1790920049.31 total 50.05 last 60.92) - open/close 4201 (time 1790920059.32 total 60.05 last 73.98) - open/close 4948 (time 1790920069.32 total 70.05 last 74.70) - open/close 5683 (time 1790920079.33 total 80.06 last 73.41) - open/close 6430 (time 1790920089.33 total 90.07 last 74.67) - open/close 7114 (time 1790920099.34 total 100.07 last 68.38) - open/close 7878 (time 1790920109.35 total 110.08 last 76.33) - open/close 8679 (time 1790920119.35 total 120.08 last 80.07) - open/close 9418 (time 1790920129.35 total 130.08 last 73.90) total: 10000 open/close in 137.91 seconds: 72.51 ops/second - unlinked 0 (time 1790920141 ; total 0 ; last 0) total: 10000 unlinks in 85 seconds: 117.647057 unlinks/second - open/close 792 (time 1790920240.97 total 10.00 last 79.19) - open/close 1559 (time 1790920250.98 total 20.01 last 76.65) - open/close 2336 (time 1790920260.98 total 30.01 last 77.66) - open/close 3099 (time 1790920270.99 total 40.02 last 76.24) - open/close 3857 (time 1790920280.99 total 50.02 last 75.78) - open/close 4510 (time 1790920291.02 total 60.06 last 65.08) - open/close 5261 (time 1790920301.04 total 70.07 last 75.01) - open/close 5997 (time 1790920311.04 total 80.08 last 73.54) - open/close 6592 (time 1790920321.05 total 90.08 last 59.46) - open/close 7287 (time 1790920331.05 total 100.09 last 69.48) - open/close 7912 (time 1790920341.06 total 110.10 last 62.44) - open/close 8634 (time 1790920351.08 total 120.11 last 72.11) - open/close 9340 (time 1790920361.08 total 130.11 last 70.58) total: 10000 open/close in 139.17 seconds: 71.85 ops/second - unlinked 0 (time 1790920373 ; total 0 ; last 0) total: 10000 unlinks in 83 seconds: 120.481926 unlinks/second - open/close 670 (time 1790920471.19 total 10.00 last 66.99) - open/close 1374 (time 1790920481.21 total 20.02 last 70.25) - open/close 2032 (time 1790920491.21 total 30.02 last 65.80) - open/close 2772 (time 1790920501.21 total 40.03 last 73.97) - open/close 3530 (time 1790920511.22 total 50.03 last 75.74) - open/close 4279 (time 1790920521.23 total 60.05 last 74.82) - open/close 5073 (time 1790920531.24 total 70.06 last 79.29) - open/close 5849 (time 1790920541.25 total 80.06 last 77.57) - open/close 6628 (time 1790920551.26 total 90.07 last 77.84) - open/close 7370 (time 1790920561.26 total 100.08 last 74.15) - open/close 8113 (time 1790920571.27 total 110.09 last 74.22) - open/close 8880 (time 1790920581.28 total 120.09 last 76.67) - open/close 9607 (time 1790920591.28 total 130.10 last 72.67) total: 10000 open/close in 135.29 seconds: 73.91 ops/second - unlinked 0 (time 1790920600 ; total 0 ; last 0) total: 10000 unlinks in 85 seconds: 117.647057 unlinks/second - open/close 720 (time 1790920700.29 total 10.01 last 71.90) - open/close 1504 (time 1790920710.30 total 20.02 last 78.33) - open/close 2238 (time 1790920720.30 total 30.02 last 73.40) - open/close 2944 (time 1790920730.31 total 40.04 last 70.51) - open/close 3731 (time 1790920740.33 total 50.05 last 78.58) - open/close 4550 (time 1790920750.33 total 60.06 last 81.85) - open/close 5273 (time 1790920760.34 total 70.06 last 72.25) - open/close 5997 (time 1790920770.35 total 80.07 last 72.32) - open/close 6749 (time 1790920780.37 total 90.09 last 75.05) - open/close 7536 (time 1790920790.38 total 100.10 last 78.62) - open/close 8179 (time 1790920800.39 total 110.11 last 64.24) - open/close 8937 (time 1790920810.39 total 120.11 last 75.80) - open/close 9660 (time 1790920820.40 total 130.12 last 72.26) total: 10000 open/close in 134.76 seconds: 74.20 ops/second - unlinked 0 (time 1790920828 ; total 0 ; last 0) total: 10000 unlinks in 84 seconds: 119.047623 unlinks/second - open/close 717 (time 1790920926.77 total 10.01 last 71.60) - open/close 1470 (time 1790920936.78 total 20.02 last 75.28) - open/close 2230 (time 1790920946.78 total 30.02 last 75.98) - open/close 2955 (time 1790920956.79 total 40.03 last 72.40) - open/close 3702 (time 1790920966.80 total 50.04 last 74.65) - open/close 4491 (time 1790920976.82 total 60.06 last 78.76) - open/close 5283 (time 1790920986.82 total 70.06 last 79.19) - open/close 6043 (time 1790920996.82 total 80.06 last 75.97) - open/close 6767 (time 1790921006.84 total 90.08 last 72.30) - open/close 7488 (time 1790921016.84 total 100.08 last 72.04) - open/close 8271 (time 1790921026.85 total 110.09 last 78.30) - open/close 9065 (time 1790921036.86 total 120.10 last 79.31) - open/close 9827 (time 1790921046.87 total 130.11 last 76.12) total: 10000 open/close in 132.33 seconds: 75.57 ops/second - unlinked 0 (time 1790921052 ; total 0 ; last 0) total: 10000 unlinks in 79 seconds: 126.582275 unlinks/second - open/close 755 (time 1790921145.43 total 10.02 last 75.36) - open/close 1524 (time 1790921155.44 total 20.02 last 76.85) - open/close 2305 (time 1790921165.44 total 30.03 last 78.09) - open/close 3003 (time 1790921175.44 total 40.03 last 69.76) - open/close 3882 (time 1790921185.46 total 50.04 last 87.80) - open/close 4718 (time 1790921195.46 total 60.05 last 83.59) - open/close 5524 (time 1790921205.47 total 70.05 last 80.54) - open/close 6288 (time 1790921215.49 total 80.07 last 76.25) - open/close 7044 (time 1790921225.49 total 90.07 last 75.59) - open/close 7869 (time 1790921235.49 total 100.07 last 82.50) - open/close 8718 (time 1790921245.50 total 110.08 last 84.83) - open/close 9511 (time 1790921255.52 total 120.10 last 79.14) total: 10000 open/close in 126.65 seconds: 78.96 ops/second - unlinked 0 (time 1790921265 ; total 0 ; last 0) total: 10000 unlinks in 77 seconds: 129.870132 unlinks/second - open/close 765 (time 1790921355.81 total 10.00 last 76.48) - open/close 1504 (time 1790921365.82 total 20.00 last 73.89) - open/close 2245 (time 1790921375.82 total 30.01 last 74.09) - open/close 3057 (time 1790921385.83 total 40.02 last 81.10) - open/close 3830 (time 1790921395.84 total 50.03 last 77.19) - open/close 4628 (time 1790921405.85 total 60.04 last 79.78) - open/close 5401 (time 1790921415.87 total 70.06 last 77.14) - open/close 6166 (time 1790921425.87 total 80.06 last 76.46) - open/close 6960 (time 1790921435.88 total 90.07 last 79.36) - open/close 7738 (time 1790921445.88 total 100.07 last 77.79) - open/close 8484 (time 1790921455.89 total 110.08 last 74.48) - open/close 9250 (time 1790921465.90 total 120.08 last 76.60) total: 10000 open/close in 129.48 seconds: 77.23 ops/second - unlinked 0 (time 1790921480 ; total 0 ; last 0) total: 10000 unlinks in 82 seconds: 121.951218 unlinks/second - open/close 553 (time 1790921577.29 total 10.01 last 55.24) - open/close 1259 (time 1790921587.30 total 20.03 last 70.50) - open/close 1973 (time 1790921597.30 total 30.03 last 71.40) - open/close 2766 (time 1790921607.31 total 40.04 last 79.21) - open/close 3507 (time 1790921617.32 total 50.04 last 74.05) - open/close 4271 (time 1790921627.33 total 60.05 last 76.36) - open/close 4988 (time 1790921637.33 total 70.05 last 71.69) - open/close 5784 (time 1790921647.33 total 80.05 last 79.57) - open/close 6489 (time 1790921657.35 total 90.07 last 70.36) - open/close 7242 (time 1790921667.36 total 100.08 last 75.24) - open/close 7934 (time 1790921677.37 total 110.09 last 69.13) - open/close 8697 (time 1790921687.38 total 120.10 last 76.24) - open/close 9435 (time 1790921697.39 total 130.11 last 73.70) total: 10000 open/close in 139.05 seconds: 71.92 ops/second - unlinked 0 (time 1790921710 ; total 0 ; last 0) total: 10000 unlinks in 83 seconds: 120.481926 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg609-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg609-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg609-server Permanent disk data: Device: lustre-ost1/ost1 Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.206.109@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.206.109@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg609-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg609-server: oleg609-server.virtnet: executing set_default_debug -1 all pdsh@oleg609-client: oleg609-server: ssh exited with exit code 1 Started lustre-OST0000 oleg609-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg609-server: oleg609-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg609-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg609-server: oleg609-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg609-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg609-client.virtnet: -o user_xattr,flock 192.168.206.109@tcp:/lustre /mnt/lustre On OST0, 20520 used inodes rc=0 pdsh@oleg609-client: oleg609-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg609-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg609-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg609-server Permanent disk data: Device: lustre-ost1/ost1 Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: mgsnode=192.168.206.109@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.206.109@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=98 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre:OST0000 checking for existing Lustre data: found Read previous values: Device: lustre-ost1/ost1 Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: autodegrade=on sys.timeout=20 mgsnode=192.168.206.109@tcp Permanent disk data: Device: lustre-ost1/ost1 Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: autodegrade=on sys.timeout=20 mgsnode=192.168.206.109@tcp autodegrade=on lustre:svname=lustre-OST0000 Writing lustre-ost1/ost1 properties lustre:autodegrade=on lustre:sys.timeout=20 lustre:mgsnode=192.168.206.109@tcp lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg609-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg609-server: oleg609-server.virtnet: executing set_default_debug -1 all pdsh@oleg609-client: oleg609-server: ssh exited with exit code 1 Started lustre-OST0000 oleg609-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg609-server: oleg609-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg609-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg609-server: oleg609-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg609-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg609-client.virtnet: -o user_xattr,flock 192.168.206.109@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg609-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg609-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg609-server stop mds service on oleg609-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg609-server stop mds service on oleg609-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg609-server unloading modules via unload_modules_local on: 'oleg609-server' oleg609-server: oleg609-server.virtnet: executing unload_modules_local oleg609-server: kmemleak disabled modules unloaded. pdsh@oleg609-client: oleg609-client: ssh exited with exit code 2 pdsh@oleg609-client: oleg609-server: ssh exited with exit code 2 pdsh@oleg609-client: oleg609-client: ssh exited with exit code 2 pdsh@oleg609-client: oleg609-server: ssh exited with exit code 2 PASS 69 (2536s) repeat= (ONLY_REPEAT=) repeat_end= (ONLY_MINUTES=) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 02:19:23 (1790921963) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (3s) Stopping clients: oleg609-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg609-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg609-client: oleg609-server: ssh exited with exit code 2 oleg609-server: oleg609-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' -rwxr-xr-x. 1 green green 1946968 Oct 1 19:15 /home/green/git/lustre-release/lustre/utils/mount.lustre -rwxr-xr-x. 1 root root 12 Jan 31 2019 /usr/sbin/mount.lustre mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre loading modules on: 'oleg609-server' oleg609-server: oleg609-server.virtnet: executing load_modules_local oleg609-server: kmemleak disabled oleg609-server: Loading modules from /home/green/git/lustre-release/lustre oleg609-server: detected 4 online CPUs by sysfs oleg609-server: MODOPTS_LIBCFS= oleg609-server: Force libcfs to create 2 CPU partitions oleg609-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg609-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg609-server: mdt/mdt options: 'mdt_enable_flr_ec=1' oleg609-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format mds2: lustre-mdt2/mdt2 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 3095 sec ========== 02:20:46 (1790922046) === conf-sanity: start cleanup 02:20:48 (1790922048) === === conf-sanity: finish cleanup 02:20:51 (1790922051) ===