-----============= acceptance-small: conf-sanity ============----- Tue Oct 14 10:25:04 EDT 2025 oleg443-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32b 32c 32newtarball 110 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg443-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg443-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg443-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg443-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg443-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg443-server oleg443-server: oleg443-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg443-server' oleg443-server: oleg443-server.virtnet: executing load_modules_local oleg443-server: Loading modules from /home/green/git/lustre-release/lustre oleg443-server: detected 4 online CPUs by sysfs oleg443-server: Force libcfs to create 2 CPU partitions oleg443-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg443-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg443-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg443-server: oleg443-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg443-client: oleg443-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg443-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg443-server: oleg443-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg443-client: oleg443-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg443-server: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 40 oleg443-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg443-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg443-server stop mds service on oleg443-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg443-server == conf-sanity test 45: long unlink handling in ptlrpcd == 10:26:36 (1760451996) start mds service on oleg443-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg443-server: oleg443-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg443-client: oleg443-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg443-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg443-server: oleg443-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg443-client: oleg443-server: ssh exited with exit code 1 Started lustre-OST0000 oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg443-client.virtnet: -o user_xattr,flock oleg443-server@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg443-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg443-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg443-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg443-server: oleg443-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg443-client: oleg443-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg443-client.virtnet: -o user_xattr,flock oleg443-server@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg443-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg443-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg443-server stop mds service on oleg443-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg443-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg443-server' oleg443-server: oleg443-server.virtnet: executing unload_modules_local oleg443-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg443-server: LNET ready to unload oleg443-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded. pdsh@oleg443-client: oleg443-client: ssh exited with exit code 2 pdsh@oleg443-client: oleg443-server: ssh exited with exit code 2 pdsh@oleg443-client: oleg443-server: ssh exited with exit code 2 pdsh@oleg443-client: oleg443-client: ssh exited with exit code 2 PASS 45 (153s) == conf-sanity test 69: replace an OST with the same index ========================================================== 10:29:09 (1760452149) start mds service on oleg443-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg443-server' oleg443-server: oleg443-server.virtnet: executing load_modules_local oleg443-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg443-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg443-server: Loading modules from /home/green/git/lustre-release/lustre oleg443-server: detected 4 online CPUs by sysfs oleg443-server: Force libcfs to create 2 CPU partitions oleg443-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg443-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg443-server: oleg443-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg443-client: oleg443-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg443-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg443-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg443-server: oleg443-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg443-client: oleg443-server: ssh exited with exit code 1 Started lustre-OST0000 oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg443-client.virtnet: -o user_xattr,flock oleg443-server@tcp:/lustre /mnt/lustre On OST0, 11872 inodes available. Want 99872. rc=0 - open/close 1908 (time 1760452198.99 total 10.00 last 190.73) - open/close 4459 (time 1760452208.99 total 20.01 last 254.98) - open/close 6767 (time 1760452219.00 total 30.01 last 230.72) - open/close 9056 (time 1760452229.00 total 40.02 last 228.80) total: 10000 open/close in 45.20 seconds: 221.22 ops/second - unlinked 0 (time 1760452235 ; total 0 ; last 0) total: 10000 unlinks in 30 seconds: 333.333344 unlinks/second - open/close 2255 (time 1760452277.07 total 10.00 last 225.46) - open/close 4784 (time 1760452287.07 total 20.01 last 252.81) - open/close 6885 (time 1760452297.07 total 30.01 last 210.06) - open/close 8660 (time 1760452307.08 total 40.01 last 177.43) total: 10000 open/close in 47.33 seconds: 211.30 ops/second - unlinked 0 (time 1760452316 ; total 0 ; last 0) total: 10000 unlinks in 28 seconds: 357.142853 unlinks/second - open/close 2266 (time 1760452355.83 total 10.00 last 226.57) - open/close 4118 (time 1760452365.83 total 20.00 last 185.14) - open/close 5759 (time 1760452375.83 total 30.01 last 164.06) - open/close 7667 (time 1760452385.83 total 40.01 last 190.77) - open/close 9655 (time 1760452395.84 total 50.01 last 198.69) total: 10000 open/close in 51.82 seconds: 192.97 ops/second - unlinked 0 (time 1760452399 ; total 0 ; last 0) total: 10000 unlinks in 26 seconds: 384.615387 unlinks/second - open/close 2207 (time 1760452437.10 total 10.00 last 220.64) - open/close 4108 (time 1760452447.10 total 20.00 last 190.08) - open/close 6153 (time 1760452457.10 total 30.01 last 204.46) - open/close 8245 (time 1760452467.10 total 40.01 last 209.19) total: 10000 open/close in 48.64 seconds: 205.59 ops/second - unlinked 0 (time 1760452477 ; total 0 ; last 0) total: 10000 unlinks in 28 seconds: 357.142853 unlinks/second - open/close 2390 (time 1760452517.22 total 10.00 last 238.95) - open/close 4562 (time 1760452527.22 total 20.00 last 217.19) - open/close 6559 (time 1760452537.22 total 30.00 last 199.68) - open/close 9115 (time 1760452547.22 total 40.01 last 255.52) total: 10000 open/close in 43.19 seconds: 231.53 ops/second - unlinked 0 (time 1760452551 ; total 0 ; last 0) total: 10000 unlinks in 22 seconds: 454.545441 unlinks/second - open/close 1945 (time 1760452585.27 total 10.00 last 194.44) - open/close 4301 (time 1760452595.28 total 20.00 last 235.57) - open/close 6665 (time 1760452605.28 total 30.01 last 236.30) - open/close 9239 (time 1760452615.28 total 40.01 last 257.39) total: 10000 open/close in 43.43 seconds: 230.23 ops/second - unlinked 0 (time 1760452620 ; total 0 ; last 0) total: 10000 unlinks in 23 seconds: 434.782623 unlinks/second - open/close 2639 (time 1760452655.42 total 10.00 last 263.88) - open/close 4793 (time 1760452665.42 total 20.01 last 215.27) - open/close 7062 (time 1760452675.43 total 30.01 last 226.80) - open/close 9344 (time 1760452685.43 total 40.01 last 228.16) total: 10000 open/close in 42.95 seconds: 232.85 ops/second - unlinked 0 (time 1760452690 ; total 0 ; last 0) total: 10000 unlinks in 26 seconds: 384.615387 unlinks/second - open/close 2243 (time 1760452728.45 total 10.00 last 224.24) - open/close 4381 (time 1760452738.45 total 20.00 last 213.80) - open/close 7223 (time 1760452748.45 total 30.00 last 284.18) total: 10000 open/close in 39.27 seconds: 254.65 ops/second - unlinked 0 (time 1760452759 ; total 0 ; last 0) total: 10000 unlinks in 20 seconds: 500.000000 unlinks/second - open/close 2433 (time 1760452791.21 total 10.00 last 243.29) - open/close 4422 (time 1760452801.21 total 20.00 last 198.86) - open/close 6790 (time 1760452811.21 total 30.00 last 236.79) - open/close 9175 (time 1760452821.21 total 40.00 last 238.45) total: 10000 open/close in 43.54 seconds: 229.67 ops/second - unlinked 0 (time 1760452826 ; total 0 ; last 0) total: 10000 unlinks in 20 seconds: 500.000000 unlinks/second - open/close 3308 (time 1760452858.05 total 10.00 last 330.78) - open/close 6465 (time 1760452868.05 total 20.00 last 315.70) - open/close 9128 (time 1760452878.06 total 30.00 last 266.23) total: 10000 open/close in 32.60 seconds: 306.77 ops/second - unlinked 0 (time 1760452882 ; total 0 ; last 0) total: 10000 unlinks in 15 seconds: 666.666687 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg443-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg443-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg443-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.204.143@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.204.143@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg443-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg443-server: oleg443-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg443-client: oleg443-server: ssh exited with exit code 1 Started lustre-OST0000 oleg443-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg443-server: oleg443-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 40 oleg443-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec mount lustre on /mnt/lustre..... Starting client: oleg443-client.virtnet: -o user_xattr,flock oleg443-server@tcp:/lustre /mnt/lustre On OST0, 291 used inodes rc=0 umount lustre on /mnt/lustre..... Stopping client oleg443-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg443-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg443-server stop mds service on oleg443-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg443-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg443-server' oleg443-server: oleg443-server.virtnet: executing unload_modules_local oleg443-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg443-server: LNET ready to unload oleg443-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded. pdsh@oleg443-client: oleg443-client: ssh exited with exit code 2 pdsh@oleg443-client: oleg443-server: ssh exited with exit code 2 pdsh@oleg443-client: oleg443-client: ssh exited with exit code 2 pdsh@oleg443-client: oleg443-server: ssh exited with exit code 2 PASS 69 (790s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 10:42:20 (1760452940) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (1s) Stopping clients: oleg443-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg443-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg443-client: oleg443-server: ssh exited with exit code 2 oleg443-server: oleg443-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg443-server' oleg443-server: oleg443-server.virtnet: executing load_modules_local oleg443-server: Loading modules from /home/green/git/lustre-release/lustre oleg443-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg443-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg443-server: detected 4 online CPUs by sysfs oleg443-server: Force libcfs to create 2 CPU partitions oleg443-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg443-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 1062 sec ========== 10:42:46 (1760452966)