-----============= acceptance-small: conf-sanity ============----- Tue Oct 7 09:12:04 EDT 2025 oleg252-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32b 32c 32newtarball 110 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg252-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg252-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg252-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg252-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg252-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg252-server oleg252-server: oleg252-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg252-server' oleg252-server: oleg252-server.virtnet: executing load_modules_local oleg252-server: Loading modules from /home/green/git/lustre-release/lustre oleg252-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg252-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg252-server: detected 4 online CPUs by sysfs oleg252-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg252-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg252-server: oleg252-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg252-client: oleg252-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg252-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg252-server: oleg252-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg252-client: oleg252-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg252-server: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 40 oleg252-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg252-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg252-server stop mds service on oleg252-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg252-server == conf-sanity test 45: long unlink handling in ptlrpcd == 09:13:23 (1759842803) start mds service on oleg252-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg252-server: oleg252-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg252-client: oleg252-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg252-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg252-server: oleg252-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg252-client: oleg252-server: ssh exited with exit code 1 Started lustre-OST0000 oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg252-client.virtnet: -o user_xattr,flock oleg252-server@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg252-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg252-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg252-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg252-server: oleg252-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg252-client: oleg252-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg252-client.virtnet: -o user_xattr,flock oleg252-server@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg252-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg252-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg252-server stop mds service on oleg252-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg252-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg252-server' oleg252-server: oleg252-server.virtnet: executing unload_modules_local oleg252-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg252-server: LNET ready to unload oleg252-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded. pdsh@oleg252-client: oleg252-client: ssh exited with exit code 2 pdsh@oleg252-client: oleg252-server: ssh exited with exit code 2 pdsh@oleg252-client: oleg252-client: ssh exited with exit code 2 pdsh@oleg252-client: oleg252-server: ssh exited with exit code 2 PASS 45 (138s) == conf-sanity test 69: replace an OST with the same index ========================================================== 09:15:42 (1759842942) start mds service on oleg252-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg252-server' oleg252-server: oleg252-server.virtnet: executing load_modules_local oleg252-server: Loading modules from /home/green/git/lustre-release/lustre oleg252-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg252-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg252-server: detected 4 online CPUs by sysfs oleg252-server: Force libcfs to create 2 CPU partitions oleg252-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg252-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg252-server: oleg252-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg252-client: oleg252-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg252-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg252-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg252-server: oleg252-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg252-client: oleg252-server: ssh exited with exit code 1 Started lustre-OST0000 oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg252-client.virtnet: -o user_xattr,flock oleg252-server@tcp:/lustre /mnt/lustre On OST0, 11872 inodes available. Want 99872. rc=0 - open/close 4813 (time 1759842978.56 total 10.00 last 481.29) - open/close 9279 (time 1759842988.56 total 20.00 last 446.55) total: 10000 open/close in 21.53 seconds: 464.55 ops/second - unlinked 0 (time 1759842991 ; total 0 ; last 0) total: 10000 unlinks in 16 seconds: 625.000000 unlinks/second - open/close 4039 (time 1759843018.48 total 10.00 last 403.86) - open/close 8328 (time 1759843028.48 total 20.00 last 428.83) total: 10000 open/close in 23.76 seconds: 420.93 ops/second - unlinked 0 (time 1759843033 ; total 0 ; last 0) total: 10000 unlinks in 14 seconds: 714.285706 unlinks/second - open/close 4071 (time 1759843058.68 total 10.00 last 407.03) - open/close 8397 (time 1759843068.68 total 20.00 last 432.51) total: 10000 open/close in 23.19 seconds: 431.20 ops/second - unlinked 0 (time 1759843073 ; total 0 ; last 0) total: 10000 unlinks in 12 seconds: 833.333313 unlinks/second - open/close 5163 (time 1759843096.78 total 10.00 last 516.27) total: 10000 open/close in 19.47 seconds: 513.74 ops/second - unlinked 0 (time 1759843107 ; total 0 ; last 0) total: 10000 unlinks in 12 seconds: 833.333313 unlinks/second - open/close 4739 (time 1759843130.53 total 10.00 last 473.84) - open/close 9182 (time 1759843140.53 total 20.00 last 444.24) total: 10000 open/close in 21.96 seconds: 455.39 ops/second - unlinked 0 (time 1759843143 ; total 0 ; last 0) total: 10000 unlinks in 15 seconds: 666.666687 unlinks/second - open/close 4057 (time 1759843170.49 total 10.00 last 405.62) - open/close 8372 (time 1759843180.49 total 20.00 last 431.41) total: 10000 open/close in 23.50 seconds: 425.50 ops/second - unlinked 0 (time 1759843185 ; total 0 ; last 0) total: 10000 unlinks in 14 seconds: 714.285706 unlinks/second - open/close 4434 (time 1759843212.15 total 10.00 last 443.31) - open/close 8853 (time 1759843222.15 total 20.00 last 441.82) total: 10000 open/close in 22.50 seconds: 444.47 ops/second - unlinked 0 (time 1759843226 ; total 0 ; last 0) total: 10000 unlinks in 14 seconds: 714.285706 unlinks/second - open/close 4224 (time 1759843252.26 total 10.00 last 422.38) - open/close 8292 (time 1759843262.26 total 20.00 last 406.73) total: 10000 open/close in 23.95 seconds: 417.45 ops/second - unlinked 0 (time 1759843267 ; total 0 ; last 0) total: 10000 unlinks in 14 seconds: 714.285706 unlinks/second - open/close 4292 (time 1759843292.79 total 10.00 last 429.14) - open/close 8878 (time 1759843302.79 total 20.00 last 458.54) total: 10000 open/close in 22.44 seconds: 445.65 ops/second - unlinked 0 (time 1759843306 ; total 0 ; last 0) total: 10000 unlinks in 14 seconds: 714.285706 unlinks/second - open/close 4547 (time 1759843331.97 total 10.00 last 454.70) - open/close 9100 (time 1759843341.97 total 20.00 last 455.21) total: 10000 open/close in 22.17 seconds: 450.96 ops/second - unlinked 0 (time 1759843345 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg252-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg252-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg252-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.202.152@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.202.152@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg252-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg252-server: oleg252-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg252-client: oleg252-server: ssh exited with exit code 1 Started lustre-OST0000 oleg252-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg252-server: oleg252-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 40 oleg252-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 9 sec mount lustre on /mnt/lustre..... Starting client: oleg252-client.virtnet: -o user_xattr,flock oleg252-server@tcp:/lustre /mnt/lustre On OST0, 291 used inodes rc=0 umount lustre on /mnt/lustre..... Stopping client oleg252-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg252-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg252-server stop mds service on oleg252-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg252-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg252-server' oleg252-server: oleg252-server.virtnet: executing unload_modules_local oleg252-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg252-server: LNET ready to unload oleg252-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded. pdsh@oleg252-client: oleg252-client: ssh exited with exit code 2 pdsh@oleg252-client: oleg252-server: ssh exited with exit code 2 pdsh@oleg252-client: oleg252-client: ssh exited with exit code 2 pdsh@oleg252-client: oleg252-server: ssh exited with exit code 2 PASS 69 (470s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 09:23:32 (1759843412) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (2s) Stopping clients: oleg252-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg252-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg252-client: oleg252-server: ssh exited with exit code 2 oleg252-server: oleg252-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg252-server' oleg252-server: oleg252-server.virtnet: executing load_modules_local oleg252-server: Loading modules from /home/green/git/lustre-release/lustre oleg252-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg252-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg252-server: detected 4 online CPUs by sysfs oleg252-server: Force libcfs to create 2 CPU partitions oleg252-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg252-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 714 sec =========== 09:23:59 (1759843439)