-----============= acceptance-small: conf-sanity ============----- Wed Jan 7 21:33:26 EST 2026 oleg439-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32b 32c 32newtarball 110 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg439-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg439-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg439-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg439-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg439-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg439-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg439-server oleg439-server: oleg439-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg439-server' oleg439-server: oleg439-server.virtnet: executing load_modules_local oleg439-server: Loading modules from /home/green/git/lustre-release/lustre oleg439-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg439-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg439-server: detected 4 online CPUs by sysfs oleg439-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format mds2: lustre-mdt2/mdt2 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg439-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg439-server: oleg439-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg439-client: oleg439-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 start mds service on oleg439-server Starting mds2: -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg439-server: oleg439-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg439-client: oleg439-server: ssh exited with exit code 1 Commit the device label on lustre-mdt2/mdt2 Started lustre-MDT0001 oleg439-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg439-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg439-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg439-server: oleg439-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg439-client: oleg439-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg439-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg439-server: oleg439-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 40 oleg439-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec oleg439-server: oleg439-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 40 oleg439-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg439-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg439-server stop mds service on oleg439-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg439-server stop mds service on oleg439-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg439-server == conf-sanity test 45: long unlink handling in ptlrpcd == 21:35:24 (1767839724) start mds service on oleg439-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg439-server: oleg439-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg439-client: oleg439-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg439-server Starting mds2: -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg439-server: oleg439-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg439-client: oleg439-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg439-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg439-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg439-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg439-server: oleg439-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg439-client: oleg439-server: ssh exited with exit code 1 Started lustre-OST0000 oleg439-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg439-client.virtnet: -o user_xattr,flock oleg439-server@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg439-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg439-server stop mds service on oleg439-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg439-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg439-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg439-server: oleg439-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg439-client: oleg439-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg439-server Starting mds2: -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg439-server: oleg439-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg439-client: oleg439-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg439-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg439-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg439-client.virtnet: -o user_xattr,flock oleg439-server@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg439-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg439-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg439-server stop mds service on oleg439-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg439-server stop mds service on oleg439-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg439-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg439-server' oleg439-server: oleg439-server.virtnet: executing unload_modules_local oleg439-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg439-server: LNET ready to unload oleg439-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded. pdsh@oleg439-client: oleg439-client: ssh exited with exit code 2 pdsh@oleg439-client: oleg439-server: ssh exited with exit code 2 pdsh@oleg439-client: oleg439-client: ssh exited with exit code 2 pdsh@oleg439-client: oleg439-server: ssh exited with exit code 2 PASS 45 (192s) == conf-sanity test 69: replace an OST with the same index ========================================================== 21:38:36 (1767839916) start mds service on oleg439-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg439-server' oleg439-server: oleg439-server.virtnet: executing load_modules_local oleg439-server: Loading modules from /home/green/git/lustre-release/lustre oleg439-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg439-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg439-server: detected 4 online CPUs by sysfs oleg439-server: Force libcfs to create 2 CPU partitions oleg439-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg439-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg439-server: oleg439-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg439-client: oleg439-server: ssh exited with exit code 1 Started lustre-MDT0000 start mds service on oleg439-server Starting mds2: -o localrecov lustre-mdt2/mdt2 /mnt/lustre-mds2 oleg439-server: oleg439-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg439-client: oleg439-server: ssh exited with exit code 1 Started lustre-MDT0001 oleg439-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid oleg439-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid start ost1 service on oleg439-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg439-server: oleg439-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg439-client: oleg439-server: ssh exited with exit code 1 Started lustre-OST0000 oleg439-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg439-client.virtnet: -o user_xattr,flock oleg439-server@tcp:/lustre /mnt/lustre On OST0, 11872 inodes available. Want 99872. rc=0 - open/close 1492 (time 1767839982.42 total 10.00 last 149.16) - open/close 2898 (time 1767839992.42 total 20.00 last 140.60) - open/close 4373 (time 1767840002.42 total 30.00 last 147.47) - open/close 6104 (time 1767840012.43 total 40.01 last 172.99) - open/close 7703 (time 1767840022.44 total 50.02 last 159.83) - open/close 9458 (time 1767840032.44 total 60.02 last 175.47) total: 10000 open/close in 63.21 seconds: 158.20 ops/second - unlinked 0 (time 1767840037 ; total 0 ; last 0) total: 10000 unlinks in 36 seconds: 277.777771 unlinks/second - open/close 1285 (time 1767840086.20 total 10.01 last 128.42) - open/close 2995 (time 1767840096.21 total 20.01 last 170.99) - open/close 4569 (time 1767840106.21 total 30.01 last 157.37) - open/close 5832 (time 1767840116.21 total 40.01 last 126.27) - open/close 7212 (time 1767840126.21 total 50.01 last 137.97) - open/close 8486 (time 1767840136.21 total 60.02 last 127.36) - open/close 9916 (time 1767840146.21 total 70.02 last 143.00) total: 10000 open/close in 70.53 seconds: 141.78 ops/second - unlinked 0 (time 1767840148 ; total 0 ; last 0) total: 10000 unlinks in 40 seconds: 250.000000 unlinks/second - open/close 1330 (time 1767840200.50 total 10.00 last 132.98) - open/close 2730 (time 1767840210.50 total 20.00 last 139.99) - open/close 4107 (time 1767840220.51 total 30.01 last 137.56) - open/close 5436 (time 1767840230.51 total 40.01 last 132.88) - open/close 7013 (time 1767840240.51 total 50.02 last 157.66) - open/close 8475 (time 1767840250.51 total 60.02 last 146.14) - open/close 9812 (time 1767840260.52 total 70.02 last 133.68) total: 10000 open/close in 71.18 seconds: 140.49 ops/second - unlinked 0 (time 1767840263 ; total 0 ; last 0) total: 10000 unlinks in 41 seconds: 243.902435 unlinks/second - open/close 1915 (time 1767840316.93 total 10.00 last 191.50) - open/close 4375 (time 1767840326.94 total 20.00 last 245.90) - open/close 6304 (time 1767840336.94 total 30.00 last 192.90) - open/close 7980 (time 1767840346.94 total 40.01 last 167.53) - open/close 9566 (time 1767840356.94 total 50.01 last 158.58) total: 10000 open/close in 52.29 seconds: 191.23 ops/second - unlinked 0 (time 1767840361 ; total 0 ; last 0) total: 10000 unlinks in 29 seconds: 344.827576 unlinks/second - open/close 4476 (time 1767840401.78 total 10.00 last 447.53) - open/close 8649 (time 1767840411.78 total 20.00 last 417.22) total: 10000 open/close in 23.36 seconds: 428.02 ops/second - unlinked 0 (time 1767840416 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second - open/close 5067 (time 1767840441.16 total 10.00 last 506.62) - open/close 9939 (time 1767840451.16 total 20.00 last 487.19) total: 10000 open/close in 20.11 seconds: 497.26 ops/second - unlinked 0 (time 1767840452 ; total 0 ; last 0) total: 10000 unlinks in 12 seconds: 833.333313 unlinks/second - open/close 5079 (time 1767840476.68 total 10.00 last 507.88) total: 10000 open/close in 19.15 seconds: 522.11 ops/second - unlinked 0 (time 1767840487 ; total 0 ; last 0) total: 10000 unlinks in 12 seconds: 833.333313 unlinks/second - open/close 4984 (time 1767840511.84 total 10.00 last 498.39) total: 10000 open/close in 19.43 seconds: 514.79 ops/second - unlinked 0 (time 1767840522 ; total 0 ; last 0) total: 10000 unlinks in 12 seconds: 833.333313 unlinks/second - open/close 5348 (time 1767840546.15 total 10.00 last 534.73) total: 10000 open/close in 18.41 seconds: 543.05 ops/second - unlinked 0 (time 1767840555 ; total 0 ; last 0) total: 10000 unlinks in 14 seconds: 714.285706 unlinks/second - open/close 4961 (time 1767840580.32 total 10.00 last 496.02) total: 10000 open/close in 19.27 seconds: 519.00 ops/second - unlinked 0 (time 1767840590 ; total 0 ; last 0) total: 10000 unlinks in 13 seconds: 769.230774 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg439-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg439-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg439-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.204.139@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.204.139@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg439-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg439-server: oleg439-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg439-client: oleg439-server: ssh exited with exit code 1 Started lustre-OST0000 oleg439-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg439-server: oleg439-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 40 oleg439-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 9 sec oleg439-server: oleg439-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 40 oleg439-server: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg439-client.virtnet: -o user_xattr,flock oleg439-server@tcp:/lustre /mnt/lustre On OST0, 5735 used inodes rc=0 umount lustre on /mnt/lustre..... Stopping client oleg439-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg439-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg439-server stop mds service on oleg439-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg439-server stop mds service on oleg439-server Stopping /mnt/lustre-mds2 (opts:-f) on oleg439-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg439-server' oleg439-server: oleg439-server.virtnet: executing unload_modules_local oleg439-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg439-server: LNET ready to unload oleg439-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded. pdsh@oleg439-client: oleg439-client: ssh exited with exit code 2 pdsh@oleg439-client: oleg439-server: ssh exited with exit code 2 pdsh@oleg439-client: oleg439-client: ssh exited with exit code 2 pdsh@oleg439-client: oleg439-server: ssh exited with exit code 2 PASS 69 (742s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 21:50:58 (1767840658) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (2s) Stopping clients: oleg439-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg439-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg439-client: oleg439-server: ssh exited with exit code 2 oleg439-server: oleg439-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg439-server' oleg439-server: oleg439-server.virtnet: executing load_modules_local oleg439-server: Loading modules from /home/green/git/lustre-release/lustre oleg439-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg439-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg439-server: detected 4 online CPUs by sysfs oleg439-server: Force libcfs to create 2 CPU partitions oleg439-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg439-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format mds2: lustre-mdt2/mdt2 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 1082 sec ========== 21:51:29 (1767840689)