-----============= acceptance-small: conf-sanity ============----- Thu Mar 26 05:37:43 EDT 2026 oleg210-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/conf-sanity.*ex': No such file or directory excepting tests: 32b 32c 32newtarball 110 108a 112a 112b 113 117 119 121 122a 123aa 123ab 123ac 123ad 123ae 123af 123ag 123ah 123ahi 123F 123G 123H 126 129 132 133 135 136 137 150 152 153a 153b 153c 155 802a Stopping clients: oleg210-client.virtnet /mnt/lustre (opts:-f) Stopping client oleg210-client.virtnet /mnt/lustre opts:-f Stopping clients: oleg210-client.virtnet /mnt/lustre2 (opts:-f) Stopping /mnt/lustre-mds1 (opts:-f) on oleg210-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg210-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg210-server oleg210-server: oleg210-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions loading modules on: 'oleg210-server' oleg210-server: oleg210-server.virtnet: executing load_modules_local oleg210-server: Loading modules from /home/green/git/lustre-release/lustre oleg210-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg210-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg210-server: detected 4 online CPUs by sysfs oleg210-server: Force libcfs to create 2 CPU partitions Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 start mds service on oleg210-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg210-server: oleg210-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg210-client: oleg210-server: ssh exited with exit code 1 Commit the device label on lustre-mdt1/mdt1 Started lustre-MDT0000 oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg210-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg210-server: oleg210-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg210-client: oleg210-server: ssh exited with exit code 1 Commit the device label on lustre-ost1/ost1 Started lustre-OST0000 oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg210-server: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 40 oleg210-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec stop ost1 service on oleg210-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg210-server stop mds service on oleg210-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg210-server == conf-sanity test 45: long unlink handling in ptlrpcd == 05:40:43 (1774518043) start mds service on oleg210-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg210-server: oleg210-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg210-client: oleg210-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg210-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg210-server: oleg210-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg210-client: oleg210-server: ssh exited with exit code 1 Started lustre-OST0000 oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg210-client.virtnet: -o user_xattr,flock oleg210-server@tcp:/lustre /mnt/lustre setup single mount lustre success stop mds service on oleg210-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg210-server sleep 60 sec fail_loc=0x8000050f sleep 10 sec manual umount lustre on /mnt/lustre.... df: /mnt/lustre: Input/output error fail_loc=0x0 start mds service on oleg210-server Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg210-server: oleg210-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg210-client: oleg210-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg210-client.virtnet: -o user_xattr,flock oleg210-server@tcp:/lustre /mnt/lustre umount lustre on /mnt/lustre..... Stopping client oleg210-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg210-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg210-server stop mds service on oleg210-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg210-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg210-server' oleg210-server: oleg210-server.virtnet: executing unload_modules_local oleg210-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg210-server: LNET ready to unload oleg210-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded. pdsh@oleg210-client: oleg210-client: ssh exited with exit code 2 pdsh@oleg210-client: oleg210-server: ssh exited with exit code 2 pdsh@oleg210-client: oleg210-client: ssh exited with exit code 2 pdsh@oleg210-client: oleg210-server: ssh exited with exit code 2 PASS 45 (221s) == conf-sanity test 69: replace an OST with the same index ========================================================== 05:44:23 (1774518263) start mds service on oleg210-server /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg210-server' oleg210-server: oleg210-server.virtnet: executing load_modules_local oleg210-server: Loading modules from /home/green/git/lustre-release/lustre oleg210-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg210-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg210-server: detected 4 online CPUs by sysfs oleg210-server: Force libcfs to create 2 CPU partitions oleg210-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg210-server: quota/lquota options: 'hash_lqs_cur_bits=3' Starting mds1: -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg210-server: oleg210-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg210-client: oleg210-server: ssh exited with exit code 1 Started lustre-MDT0000 oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid start ost1 service on oleg210-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg210-server: oleg210-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg210-client: oleg210-server: ssh exited with exit code 1 Started lustre-OST0000 oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid mount lustre on /mnt/lustre..... Starting client: oleg210-client.virtnet: -o user_xattr,flock oleg210-server@tcp:/lustre /mnt/lustre On OST0, 11872 inodes available. Want 99872. rc=0 - open/close 883 (time 1774518348.79 total 10.01 last 88.22) - open/close 1533 (time 1774518358.81 total 20.02 last 64.92) - open/close 2347 (time 1774518368.81 total 30.02 last 81.39) - open/close 2921 (time 1774518378.83 total 40.04 last 57.29) - open/close 3446 (time 1774518388.83 total 50.04 last 52.49) - open/close 4115 (time 1774518398.84 total 60.05 last 66.83) - open/close 4683 (time 1774518408.84 total 70.06 last 56.79) - open/close 5419 (time 1774518418.89 total 80.10 last 73.27) - open/close 6283 (time 1774518428.89 total 90.10 last 86.37) - open/close 7138 (time 1774518438.89 total 100.11 last 85.49) - open/close 7864 (time 1774518448.90 total 110.12 last 72.53) - open/close 8772 (time 1774518458.92 total 120.13 last 90.66) - open/close 9602 (time 1774518468.92 total 130.13 last 82.99) total: 10000 open/close in 134.37 seconds: 74.42 ops/second - unlinked 0 (time 1774518477 ; total 0 ; last 0) total: 10000 unlinks in 67 seconds: 149.253738 unlinks/second - open/close 973 (time 1774518558.55 total 10.00 last 97.28) - open/close 1875 (time 1774518568.55 total 20.00 last 90.18) - open/close 2909 (time 1774518578.56 total 30.01 last 103.35) - open/close 3840 (time 1774518588.57 total 40.02 last 92.99) - open/close 4889 (time 1774518598.59 total 50.04 last 104.70) - open/close 5872 (time 1774518608.59 total 60.04 last 98.29) - open/close 6882 (time 1774518618.59 total 70.04 last 100.97) - open/close 7873 (time 1774518628.60 total 80.05 last 99.04) - open/close 8827 (time 1774518638.60 total 90.05 last 95.36) - open/close 9869 (time 1774518648.61 total 100.06 last 104.13) total: 10000 open/close in 101.38 seconds: 98.63 ops/second - unlinked 0 (time 1774518653 ; total 0 ; last 0) total: 10000 unlinks in 50 seconds: 200.000000 unlinks/second - open/close 1132 (time 1774518716.92 total 10.01 last 113.14) - open/close 2200 (time 1774518726.92 total 20.01 last 106.76) - open/close 3320 (time 1774518736.92 total 30.01 last 111.96) - open/close 5016 (time 1774518746.94 total 40.02 last 169.41) - open/close 6317 (time 1774518756.94 total 50.03 last 130.01) - open/close 7440 (time 1774518766.96 total 60.05 last 112.12) - open/close 8697 (time 1774518776.96 total 70.05 last 125.70) total: 10000 open/close in 78.37 seconds: 127.59 ops/second - unlinked 0 (time 1774518787 ; total 0 ; last 0) total: 10000 unlinks in 39 seconds: 256.410248 unlinks/second - open/close 1688 (time 1774518838.12 total 10.00 last 168.73) - open/close 3873 (time 1774518848.12 total 20.00 last 218.50) - open/close 5979 (time 1774518858.13 total 30.01 last 210.43) - open/close 7417 (time 1774518868.13 total 40.02 last 143.75) - open/close 8794 (time 1774518878.14 total 50.02 last 137.62) total: 10000 open/close in 59.95 seconds: 166.80 ops/second - unlinked 0 (time 1774518889 ; total 0 ; last 0) total: 10000 unlinks in 40 seconds: 250.000000 unlinks/second - open/close 1529 (time 1774518941.85 total 10.00 last 152.89) - open/close 3031 (time 1774518951.85 total 20.00 last 150.15) - open/close 4338 (time 1774518961.86 total 30.01 last 130.63) - open/close 5649 (time 1774518971.87 total 40.01 last 131.02) - open/close 6862 (time 1774518981.87 total 50.02 last 121.20) - open/close 8068 (time 1774518991.88 total 60.02 last 120.58) - open/close 9283 (time 1774519001.88 total 70.03 last 121.48) total: 10000 open/close in 75.02 seconds: 133.30 ops/second - unlinked 0 (time 1774519008 ; total 0 ; last 0) total: 10000 unlinks in 38 seconds: 263.157898 unlinks/second - open/close 1440 (time 1774519058.09 total 10.01 last 143.85) - open/close 2784 (time 1774519068.09 total 20.01 last 134.37) - open/close 4081 (time 1774519078.10 total 30.02 last 129.66) - open/close 5437 (time 1774519088.10 total 40.02 last 135.50) - open/close 6759 (time 1774519098.11 total 50.03 last 132.14) - open/close 8152 (time 1774519108.11 total 60.03 last 139.26) - open/close 9651 (time 1774519118.11 total 70.03 last 149.89) total: 10000 open/close in 72.30 seconds: 138.31 ops/second - unlinked 0 (time 1774519122 ; total 0 ; last 0) total: 10000 unlinks in 40 seconds: 250.000000 unlinks/second - open/close 1275 (time 1774519174.81 total 10.00 last 127.45) - open/close 2526 (time 1774519184.81 total 20.01 last 125.05) - open/close 3782 (time 1774519194.81 total 30.01 last 125.59) - open/close 4966 (time 1774519204.81 total 40.01 last 118.37) - open/close 5987 (time 1774519214.82 total 50.02 last 102.06) - open/close 6998 (time 1774519224.83 total 60.02 last 101.01) - open/close 8373 (time 1774519234.83 total 70.03 last 137.48) - open/close 9830 (time 1774519244.83 total 80.03 last 145.62) total: 10000 open/close in 81.21 seconds: 123.13 ops/second - unlinked 0 (time 1774519248 ; total 0 ; last 0) total: 10000 unlinks in 35 seconds: 285.714294 unlinks/second - open/close 1561 (time 1774519295.76 total 10.00 last 156.10) - open/close 3170 (time 1774519305.76 total 20.00 last 160.87) - open/close 4555 (time 1774519315.77 total 30.01 last 138.42) - open/close 5809 (time 1774519325.77 total 40.01 last 125.37) - open/close 7276 (time 1774519335.78 total 50.02 last 146.56) - open/close 8622 (time 1774519345.78 total 60.02 last 134.56) - open/close 9943 (time 1774519355.79 total 70.03 last 131.95) total: 10000 open/close in 70.40 seconds: 142.05 ops/second - unlinked 0 (time 1774519358 ; total 0 ; last 0) total: 10000 unlinks in 39 seconds: 256.410248 unlinks/second - open/close 1459 (time 1774519409.60 total 10.00 last 145.87) - open/close 3218 (time 1774519419.60 total 20.00 last 175.87) - open/close 5051 (time 1774519429.61 total 30.01 last 183.17) - open/close 6585 (time 1774519439.61 total 40.01 last 153.40) - open/close 8262 (time 1774519449.61 total 50.02 last 167.62) - open/close 9956 (time 1774519459.61 total 60.02 last 169.39) total: 10000 open/close in 60.26 seconds: 165.96 ops/second - unlinked 0 (time 1774519461 ; total 0 ; last 0) total: 10000 unlinks in 32 seconds: 312.500000 unlinks/second - open/close 1658 (time 1774519505.57 total 10.01 last 165.69) - open/close 3102 (time 1774519515.57 total 20.01 last 144.31) - open/close 4322 (time 1774519525.58 total 30.02 last 121.94) - open/close 5705 (time 1774519535.58 total 40.02 last 138.27) - open/close 7175 (time 1774519545.58 total 50.02 last 146.97) - open/close 8595 (time 1774519555.59 total 60.03 last 141.94) - open/close 9977 (time 1774519565.59 total 70.03 last 138.15) total: 10000 open/close in 70.20 seconds: 142.46 ops/second - unlinked 0 (time 1774519567 ; total 0 ; last 0) total: 10000 unlinks in 36 seconds: 277.777771 unlinks/second rm: missing operand Try 'rm --help' for more information. umount lustre on /mnt/lustre..... Stopping client oleg210-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg210-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg210-server Permanent disk data: Target: lustre-OST0000 Index: 0 Lustre FS: lustre Mount type: zfs Flags: 0x42 (OST update ) Persistent mount opts: Parameters: mgsnode=192.168.202.110@tcp sys.timeout=20 autodegrade=on mkfs_cmd = zpool create -o feature@longname=disabled -f -O canmount=off lustre-ost1 /dev/vde mkfs_cmd = zfs create -o canmount=off -o quota=409600000 lustre-ost1/ost1 xattr=sa dnodesize=auto recordsize=1M compression=off Writing lustre-ost1/ost1 properties lustre:mgsnode=192.168.202.110@tcp lustre:sys.timeout=20 lustre:autodegrade=on lustre:version=1 lustre:flags=66 lustre:index=0 lustre:fsname=lustre lustre:svname=lustre-OST0000 start ost1 service on oleg210-server Starting ost1: -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 oleg210-server: oleg210-server.virtnet: executing set_default_debug -1 all 8 pdsh@oleg210-client: oleg210-server: ssh exited with exit code 1 Started lustre-OST0000 oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid oleg210-server: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 40 oleg210-server: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec mount lustre on /mnt/lustre..... Starting client: oleg210-client.virtnet: -o user_xattr,flock oleg210-server@tcp:/lustre /mnt/lustre On OST0, 10351 used inodes rc=0 umount lustre on /mnt/lustre..... Stopping client oleg210-client.virtnet /mnt/lustre (opts:) stop ost1 service on oleg210-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg210-server stop mds service on oleg210-server Stopping /mnt/lustre-mds1 (opts:-f) on oleg210-server /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy unloading modules on: 'oleg210-server' oleg210-server: oleg210-server.virtnet: executing unload_modules_local oleg210-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy oleg210-server: LNET ready to unload oleg210-server: /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy modules unloaded. pdsh@oleg210-client: oleg210-client: ssh exited with exit code 2 pdsh@oleg210-client: oleg210-server: ssh exited with exit code 2 pdsh@oleg210-client: oleg210-client: ssh exited with exit code 2 pdsh@oleg210-client: oleg210-server: ssh exited with exit code 2 PASS 69 (1388s) == conf-sanity test 111: Adding large_dir with over 2GB directory ========================================================== 06:07:32 (1774519652) SKIP: conf-sanity test_111 Only applicable to ldiskfs-based MDTs SKIP 111 (2s) Stopping clients: oleg210-client.virtnet /mnt/lustre (opts:-f) Stopping clients: oleg210-client.virtnet /mnt/lustre2 (opts:-f) pdsh@oleg210-client: oleg210-server: ssh exited with exit code 2 oleg210-server: oleg210-server.virtnet: executing set_hostid /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg210-server' oleg210-server: oleg210-server.virtnet: executing load_modules_local oleg210-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg210-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg210-server: Loading modules from /home/green/git/lustre-release/lustre oleg210-server: detected 4 online CPUs by sysfs oleg210-server: Force libcfs to create 2 CPU partitions oleg210-server: ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' oleg210-server: quota/lquota options: 'hash_lqs_cur_bits=3' Formatting mgs, mds, osts Format mds1: lustre-mdt1/mdt1 Format ost1: lustre-ost1/ost1 Format ost2: lustre-ost2/ost2 == conf-sanity test complete, duration 1827 sec ========== 06:08:11 (1774519691)