-----============= acceptance-small: conf-sanity ============----- Tue Sep 29 00:56:03 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky use CONF_SANITY_ONLY='' CONF_SANITY_EXCEPT='' oleg124-server: /home/green/git/lustre-release/lustre/tests/except/conf-sanity.ex - see CLIENT_VERSION >= v2_15_63-53-g1a6ef725c2 (34683455 >= 34553653) for LU-16938, go 81 - see CLIENT_VERSION >= v2_15_65-31-g74a5b9efaa (34683455 >= 34554143) for LU-18348, go 32b 32c - see CLIENT_VERSION >= v2_16_55-19-gf69f27cc35 (34683455 >= 34617107) for LU-17920, go 123ae 123ag - see CLIENT_VERSION >= v2_17_52-94-g86904ef943 (34683455 >= 34681950) for LU-19740, go 115 only running tests: 45 69 111 excepting tests: 32b 32c 32newtarball 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg124-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg124-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg124-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg124-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg124-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg124-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg124-server oleg124-server: oleg124-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: kmemleak disabled oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions oleg124-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format mds2: lustre-mdt2/mdt2 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg124-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 start mds service on oleg124-server Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Commit the device label on lustre-mdt2/mdt2 Started lustre-MDT0001 oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg124-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg124-server: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg124-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg124-server: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg124-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg124-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg124-server stop mds service on oleg124-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg124-server stop mds service on oleg124-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg124-server repeat= (ONLY_REPEAT=) repeat_end= (ONLY_MINUTES=) == conf-sanity test 45: long unlink handling in ptlrpcd == 01:00:09 (1790658009) start mds service on oleg124-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg124-server Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg124-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-OST0000 oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg124-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg124-server stop mds service on oleg124-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg124-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg124-server Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg124-server Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg124-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg124-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg124-server stop mds service on oleg124-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg124-server stop mds service on oleg124-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg124-server unloading modules via unload_modules_local on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing unload_modules_local oleg124-server: kmemleak disabled modules unloaded. pdsh@oleg124-client: oleg124-server: ssh exited with exit code 2 pdsh@oleg124-client: oleg124-client: ssh exited with exit code 2 pdsh@oleg124-client: oleg124-client: ssh exited with exit code 2 pdsh@oleg124-client: oleg124-server: ssh exited with exit code 2 PASS 45 (308s) repeat= (ONLY_REPEAT=) repeat_end= (ONLY_MINUTES=) == conf-sanity test 69: replace an OST with the same index ========================================================== 01:05:18 (1790658318) start mds service on oleg124-server Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' -rwxr-xr-x. 1 green green 1940568 Sep 28 12:40 /home/green/git/lustre-release/lustre/utils/mount.lustre -rwxr-xr-x. 1 root root 12 Jan 31 2019 /usr/sbin/mount.lustre mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: kmemleak disabled oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions oleg124-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg124-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg124-server: mdt/mdt options: 'mdt_enable_flr_ec=1' oleg124-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg124-server Start mds2: mount -t lustre -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 start ost1 service on oleg124-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-OST0000 oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 mount lustre on /mnt/lustre..... Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre seq.cli-lustre-OST0000-super.width=0x1ffffff On OST0, 11904 inodes available. Want 99872. rc=0 - open/close 601 (time 1790658437.06 total 10.01 last 60.05) - open/close 1306 (time 1790658447.06 total 20.01 last 70.49) - open/close 2044 (time 1790658457.07 total 30.02 last 73.70) - open/close 2725 (time 1790658467.07 total 40.02 last 68.09) - open/close 3393 (time 1790658477.08 total 50.03 last 66.77) - open/close 4136 (time 1790658487.08 total 60.04 last 74.25) - open/close 4797 (time 1790658497.09 total 70.05 last 66.03) - open/close 5572 (time 1790658507.11 total 80.06 last 77.38) - open/close 6255 (time 1790658517.12 total 90.07 last 68.25) - open/close 7017 (time 1790658527.13 total 100.08 last 76.12) - open/close 7712 (time 1790658537.13 total 110.08 last 69.48) - open/close 8334 (time 1790658547.13 total 120.09 last 62.18) - open/close 8964 (time 1790658557.14 total 130.10 last 62.93) - open/close 9570 (time 1790658567.16 total 140.11 last 60.53) total: 10000 open/close in 146.83 seconds: 68.11 ops/second - unlinked 0 (time 1790658578 ; total 0 ; last 0) total: 10000 unlinks in 93 seconds: 107.526878 unlinks/second - open/close 685 (time 1790658686.18 total 10.01 last 68.47) - open/close 1223 (time 1790658696.18 total 20.01 last 53.80) - open/close 1982 (time 1790658706.19 total 30.01 last 75.84) - open/close 2740 (time 1790658716.19 total 40.02 last 75.77) - open/close 3453 (time 1790658726.20 total 50.03 last 71.22) - open/close 4139 (time 1790658736.21 total 60.04 last 68.56) - open/close 5005 (time 1790658746.23 total 70.05 last 86.47) - open/close 5800 (time 1790658756.23 total 80.06 last 79.45) - open/close 6496 (time 1790658766.25 total 90.07 last 69.50) - open/close 7164 (time 1790658776.25 total 100.08 last 66.74) - open/close 7895 (time 1790658786.26 total 110.09 last 73.03) - open/close 8559 (time 1790658796.27 total 120.10 last 66.34) - open/close 9327 (time 1790658806.28 total 130.11 last 76.70) - open/close 9995 (time 1790658816.29 total 140.12 last 66.74) total: 10000 open/close in 140.20 seconds: 71.33 ops/second - unlinked 0 (time 1790658821 ; total 0 ; last 0) total: 10000 unlinks in 83 seconds: 120.481926 unlinks/second - open/close 636 (time 1790658920.44 total 10.00 last 63.57) - open/close 1312 (time 1790658930.44 total 20.01 last 67.58) - open/close 2054 (time 1790658940.45 total 30.01 last 74.18) - open/close 2722 (time 1790658950.45 total 40.01 last 66.76) - open/close 3351 (time 1790658960.46 total 50.02 last 62.86) - open/close 4113 (time 1790658970.47 total 60.03 last 76.13) - open/close 4735 (time 1790658980.48 total 70.04 last 62.11) - open/close 5404 (time 1790658990.48 total 80.05 last 66.89) - open/close 6087 (time 1790659000.49 total 90.06 last 68.23) - open/close 6679 (time 1790659010.50 total 100.07 last 59.13) - open/close 7273 (time 1790659020.51 total 110.08 last 59.34) - open/close 7980 (time 1790659030.52 total 120.08 last 70.67) - open/close 8717 (time 1790659040.53 total 130.09 last 73.64) - open/close 9430 (time 1790659050.53 total 140.09 last 71.27) total: 10000 open/close in 149.77 seconds: 66.77 ops/second - unlinked 0 (time 1790659065 ; total 0 ; last 0) total: 10000 unlinks in 82 seconds: 121.951218 unlinks/second - open/close 685 (time 1790659162.98 total 10.00 last 68.49) - open/close 1551 (time 1790659172.98 total 20.00 last 86.58) - open/close 2321 (time 1790659182.98 total 30.00 last 77.00) - open/close 3222 (time 1790659192.99 total 40.01 last 90.05) - open/close 3985 (time 1790659202.99 total 50.01 last 76.30) - open/close 4825 (time 1790659212.99 total 60.01 last 83.98) - open/close 5651 (time 1790659223.00 total 70.02 last 82.52) - open/close 6387 (time 1790659233.02 total 80.04 last 73.48) - open/close 7000 (time 1790659243.03 total 90.04 last 61.25) - open/close 7764 (time 1790659253.03 total 100.05 last 76.37) - open/close 8584 (time 1790659263.04 total 110.06 last 81.92) - open/close 9335 (time 1790659273.05 total 120.06 last 75.05) total: 10000 open/close in 128.79 seconds: 77.64 ops/second - unlinked 0 (time 1790659285 ; total 0 ; last 0) total: 10000 unlinks in 86 seconds: 116.279068 unlinks/second - open/close 668 (time 1790659386.46 total 10.01 last 66.74) - open/close 1364 (time 1790659396.46 total 20.01 last 69.59) - open/close 2125 (time 1790659406.47 total 30.01 last 76.09) - open/close 2770 (time 1790659416.48 total 40.03 last 64.40) - open/close 3477 (time 1790659426.49 total 50.04 last 70.64) - open/close 4292 (time 1790659436.49 total 60.04 last 81.50) - open/close 5010 (time 1790659446.50 total 70.04 last 71.75) - open/close 5465 (time 1790659456.50 total 80.04 last 45.50) - open/close 6173 (time 1790659466.50 total 90.05 last 70.77) - open/close 6802 (time 1790659476.51 total 100.05 last 62.87) - open/close 7394 (time 1790659486.52 total 110.06 last 59.13) - open/close 8231 (time 1790659496.52 total 120.07 last 83.66) - open/close 9113 (time 1790659506.53 total 130.08 last 88.10) - open/close 9815 (time 1790659516.54 total 140.08 last 70.18) total: 10000 open/close in 142.81 seconds: 70.02 ops/second - unlinked 0 (time 1790659524 ; total 0 ; last 0) total: 10000 unlinks in 73 seconds: 136.986298 unlinks/second - open/close 887 (time 1790659613.00 total 10.02 last 88.54) - open/close 1515 (time 1790659623.01 total 20.03 last 62.71) - open/close 2167 (time 1790659633.03 total 30.05 last 65.09) - open/close 2910 (time 1790659643.04 total 40.06 last 74.20) - open/close 3731 (time 1790659653.05 total 50.07 last 82.06) - open/close 4437 (time 1790659663.05 total 60.07 last 70.56) - open/close 5252 (time 1790659673.06 total 70.08 last 81.42) - open/close 5918 (time 1790659683.08 total 80.10 last 66.51) - open/close 6606 (time 1790659693.09 total 90.11 last 68.69) - open/close 7338 (time 1790659703.09 total 100.11 last 73.19) - open/close 7897 (time 1790659713.12 total 110.14 last 55.76) - open/close 8524 (time 1790659723.12 total 120.14 last 62.68) - open/close 9383 (time 1790659733.14 total 130.16 last 85.78) total: 10000 open/close in 137.40 seconds: 72.78 ops/second - unlinked 0 (time 1790659745 ; total 0 ; last 0) total: 10000 unlinks in 91 seconds: 109.890106 unlinks/second - open/close 791 (time 1790659852.06 total 10.01 last 78.99) - open/close 1550 (time 1790659862.07 total 20.02 last 75.84) - open/close 2222 (time 1790659872.07 total 30.02 last 67.19) - open/close 2972 (time 1790659882.08 total 40.03 last 74.97) - open/close 3528 (time 1790659892.08 total 50.03 last 55.60) - open/close 4282 (time 1790659902.08 total 60.03 last 75.37) - open/close 5023 (time 1790659912.09 total 70.04 last 74.07) - open/close 5802 (time 1790659922.10 total 80.05 last 77.82) - open/close 6717 (time 1790659932.10 total 90.05 last 91.47) - open/close 7393 (time 1790659942.10 total 100.05 last 67.57) - open/close 8013 (time 1790659952.11 total 110.06 last 61.97) - open/close 8682 (time 1790659962.11 total 120.06 last 66.90) - open/close 9436 (time 1790659972.12 total 130.07 last 75.32) total: 10000 open/close in 137.44 seconds: 72.76 ops/second - unlinked 0 (time 1790659984 ; total 0 ; last 0) total: 10000 unlinks in 92 seconds: 108.695656 unlinks/second - open/close 803 (time 1790660092.09 total 10.01 last 80.19) - open/close 1552 (time 1790660102.10 total 20.03 last 74.77) - open/close 2285 (time 1790660112.11 total 30.04 last 73.23) - open/close 2915 (time 1790660122.12 total 40.04 last 62.99) - open/close 3549 (time 1790660132.12 total 50.04 last 63.40) - open/close 4184 (time 1790660142.16 total 60.09 last 63.22) - open/close 4786 (time 1790660152.18 total 70.11 last 60.07) - open/close 5504 (time 1790660162.19 total 80.12 last 71.73) - open/close 6156 (time 1790660172.22 total 90.15 last 65.02) - open/close 6795 (time 1790660182.23 total 100.15 last 63.84) - open/close 7402 (time 1790660192.27 total 110.19 last 60.46) - open/close 8045 (time 1790660202.28 total 120.21 last 64.22) - open/close 8770 (time 1790660212.29 total 130.22 last 72.41) - open/close 9431 (time 1790660222.30 total 140.23 last 66.04) total: 10000 open/close in 146.93 seconds: 68.06 ops/second - unlinked 0 (time 1790660234 ; total 0 ; last 0) total: 10000 unlinks in 78 seconds: 128.205124 unlinks/second - open/close 746 (time 1790660327.56 total 10.01 last 74.54) - open/close 1548 (time 1790660337.56 total 20.01 last 80.15) - open/close 2430 (time 1790660347.57 total 30.02 last 88.18) - open/close 3275 (time 1790660357.57 total 40.02 last 84.49) - open/close 3913 (time 1790660367.57 total 50.02 last 63.79) - open/close 4664 (time 1790660377.57 total 60.02 last 75.09) - open/close 5383 (time 1790660387.58 total 70.03 last 71.85) - open/close 6109 (time 1790660397.58 total 80.03 last 72.59) - open/close 6772 (time 1790660407.60 total 90.05 last 66.15) - open/close 7446 (time 1790660417.62 total 100.07 last 67.27) - open/close 8093 (time 1790660427.62 total 110.07 last 64.70) - open/close 8726 (time 1790660437.64 total 120.09 last 63.18) - open/close 9404 (time 1790660447.66 total 130.11 last 67.67) - open/close 9990 (time 1790660457.67 total 140.12 last 58.57) total: 10000 open/close in 140.26 seconds: 71.30 ops/second - unlinked 0 (time 1790660463 ; total 0 ; last 0) total: 10000 unlinks in 84 seconds: 119.047623 unlinks/second - open/close 595 (time 1790660562.76 total 10.00 last 59.50) - open/close 1203 (time 1790660572.77 total 20.01 last 60.76) - open/close 1877 (time 1790660582.78 total 30.02 last 67.29) - open/close 2658 (time 1790660592.78 total 40.02 last 78.09) - open/close 3348 (time 1790660602.79 total 50.03 last 68.99) - open/close 4022 (time 1790660612.79 total 60.03 last 67.39) - open/close 4775 (time 1790660622.79 total 70.03 last 75.25) - open/close 5496 (time 1790660632.80 total 80.04 last 72.04) - open/close 6219 (time 1790660642.81 total 90.05 last 72.27) - open/close 7069 (time 1790660652.81 total 100.05 last 84.94) - open/close 7943 (time 1790660662.82 total 110.06 last 87.33) - open/close 8691 (time 1790660672.82 total 120.06 last 74.79) - open/close 9361 (time 1790660682.84 total 130.08 last 66.89) - open/close 9931 (time 1790660692.84 total 140.08 last 56.99) total: 10000 open/close in 141.07 seconds: 70.88 ops/second - unlinked 0 (time 1790660699 ; total 0 ; last 0) total: 10000 unlinks in 89 seconds: 112.359550 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg124-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg124-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg124-server Permanent disk data: Device: lustre-ost1/ost1 Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.201.124@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.201.124@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg124-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-OST0000 oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg124-server: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg124-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg124-server: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg124-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre On OST0, 20520 used inodes rc=0 pdsh@oleg124-client: oleg124-server: ssh exited with exit code 22 umount lustre on /mnt/lustre..... Stopping client oleg124-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg124-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg124-server Permanent disk data: Device: lustre-ost1/ost1 Target: lustre:OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: mgsnode=192.168.201.124@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.201.124@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=98 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre:OST0000 checking for existing Lustre data: found Read previous values: Device: lustre-ost1/ost1 Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x62 (OST first_time update ) Persistent mount opts: Parameters: mgsnode=192.168.201.124@tcp sys.timeout=20 autodegrade=on Permanent disk data: Device: lustre-ost1/ost1 Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.201.124@tcp sys.timeout=20 autodegrade=on autodegrade=on lustre:svname=lustre-OST0000 Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.201.124@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg124-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg124-server: oleg124-server.virtnet: executing set_default_debug -1 all pdsh@oleg124-client: oleg124-server: ssh exited with exit code 1 Started lustre-OST0000 oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 oleg124-server: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 oleg124-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg124-server: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 oleg124-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg124-client.virtnet: -o user_xattr,flock 192.168.201.124@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg124-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg124-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg124-server stop mds service on oleg124-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg124-server stop mds service on oleg124-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg124-server unloading modules via unload_modules_local on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing unload_modules_local oleg124-server: kmemleak disabled modules unloaded. pdsh@oleg124-client: oleg124-client: ssh exited with exit code 2 pdsh@oleg124-client: oleg124-server: ssh exited with exit code 2 pdsh@oleg124-client: oleg124-client: ssh exited with exit code 2 pdsh@oleg124-client: oleg124-server: ssh exited with exit code 2 PASS 69 (2661s) repeat= (ONLY_REPEAT=) repeat_end= (ONLY_MINUTES=) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 01:49:41 (1790660981) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (4s) Stopping clients: oleg124-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg124-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg124-client: oleg124-server: ssh exited with exit code 2 oleg124-server: oleg124-server.virtnet: executing set_hostid Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' -rwxr-xr-x. 1 green green 1940568 Sep 28 12:40 /home/green/git/lustre-release/lustre/utils/mount.lustre -rwxr-xr-x. 1 root root 12 Jan 31 2019 /usr/sbin/mount.lustre mount --bind /home/green/git/lustre-release/lustre/utils/mount.lustre /usr/sbin/mount.lustre loading modules on: 'oleg124-server' oleg124-server: oleg124-server.virtnet: executing load_modules_local oleg124-server: kmemleak disabled oleg124-server: Loading modules from /home/green/git/lustre-release/lustre oleg124-server: detected 4 online CPUs by sysfs oleg124-server: MODOPTS_LIBCFS= oleg124-server: Force libcfs to create 2 CPU partitions oleg124-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg124-server: quota/lquota options: 'hash_lqs_cur_bits=3' oleg124-server: mdt/mdt options: 'mdt_enable_flr_ec=1' oleg124-server: /usr/sbin/ not writable for /usr/sbin/mount.lustre_tgt Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format mds2: lustre-mdt2/mdt2 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 3331 sec ========== 01:51:36 (1790661096) === conf-sanity: start cleanup 01:51:38 (1790661098) === === conf-sanity: finish cleanup 01:51:42 (1790661102) ===