-----============= acceptance-small: recovery-small ============----- Mon Aug 31 14:31:21 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg145-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/recovery-small.*ex': No such file or directory excepting tests: 136 === recovery-small: start setup 14:31:44 (1788201104) === oleg145-client.virtnet: executing check_config_client /mnt/lustre oleg145-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg145-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffffa026923be000.idle_timeout=debug osc.lustre-OST0001-osc-ffffa026923be000.idle_timeout=debug disable quota as required oleg145-server: oleg145-server.virtnet: executing set_default_debug -1 all === recovery-small: finish setup 14:32:23 (1788201143) === == recovery-small test 1: create, chmod, stat: drop req, drop rep ========================================================== 14:32:25 (1788201145) fail_val=0 fail_loc=0x123 fail_loc=0 fail_loc=0x119 fail_loc=0 fail_val=0 fail_loc=0x123 fail_loc=0 fail_loc=0x119 fail_loc=0 fail_val=0 fail_loc=0x123 fail_loc=0 fail_loc=0x122 fail_loc=0 PASS 1 (122s) == recovery-small test 4: open: drop req, drop rep ======= 14:34:27 (1788201267) fail_val=0 fail_loc=0x123 fail_loc=0 fail_loc=0x122 fail_loc=0 PASS 4 (30s) == recovery-small test 5: rename: drop req, drop rep ===== 14:34:57 (1788201297) fail_val=0 fail_loc=0x123 fail_loc=0 fail_loc=0x119 fail_loc=0 PASS 5 (43s) == recovery-small test 6: link, unlink: drop req, drop rep ========================================================== 14:35:40 (1788201340) fail_val=0 fail_loc=0x123 fail_loc=0 fail_loc=0x119 fail_loc=0 fail_val=0 fail_loc=0x123 fail_loc=0 fail_loc=0x119 fail_loc=0 PASS 6 (82s) == recovery-small test 8: touch: drop rep (bug 1423) ===== 14:37:02 (1788201422) fail_loc=0x119 fail_loc=0 PASS 8 (25s) == recovery-small test 9: pause bulk on OST (bug 1420) === 14:37:27 (1788201447) timeout is 0 fail_val=0 fail_loc=0x80000214 fail_loc=0 PASS 9 (16s) == recovery-small test 10a: finish request on server after client eviction (bug 1521) ========================================================== 14:37:44 (1788201464) ldlm.namespaces.lustre-MDT0000-mdc-ffffa026923be000.lru_max_age=3900s console_ratelimit=0 ldlm.namespaces.MGC192.168.201.145@tcp.early_lock_cancel=0 ldlm.namespaces.lustre-MDT0000-mdc-ffffa026923be000.early_lock_cancel=0 ldlm.namespaces.lustre-OST0000-osc-ffffa026923be000.early_lock_cancel=0 ldlm.namespaces.lustre-OST0001-osc-ffffa026923be000.early_lock_cancel=0 fail_loc=0x305 fail_loc=0 fail_val=0 ldlm.namespaces.MGC192.168.201.145@tcp.early_lock_cancel=1 ldlm.namespaces.lustre-MDT0000-mdc-ffffa026923be000.early_lock_cancel=1 ldlm.namespaces.lustre-OST0000-osc-ffffa026923be000.early_lock_cancel=1 ldlm.namespaces.lustre-OST0001-osc-ffffa026923be000.early_lock_cancel=1 /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4316: /mnt/lustre/recon: Input/output error Connected clients: oleg145-client.virtnet /mnt/lustre has perms 0777 OK [ 644.063258] Lustre: mdt00_001: service thread pid 5820 was inactive for 42.096 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [ 713.822872] Lustre: mdt00_001: service thread pid 5820 completed after 111.855s. This likely indicates the system was overloaded (too many service threads, or not enough hardware resources). ldlm.namespaces.lustre-MDT0000-mdc-ffffa026923be000.lru_max_age=600000 console_ratelimit=1 PASS 10a (125s) == recovery-small test 10b: re-send BL AST =============== 14:39:48 (1788201588) ldlm.namespaces.MGC192.168.201.145@tcp.early_lock_cancel=0 ldlm.namespaces.lustre-MDT0000-mdc-ffffa026923be000.early_lock_cancel=0 ldlm.namespaces.lustre-OST0000-osc-ffffa026923be000.early_lock_cancel=0 ldlm.namespaces.lustre-OST0001-osc-ffffa026923be000.early_lock_cancel=0 fail_loc=0x80000305 fail_loc=0 fail_val=0 ldlm.namespaces.MGC192.168.201.145@tcp.early_lock_cancel=1 ldlm.namespaces.lustre-MDT0000-mdc-ffffa026923be000.early_lock_cancel=1 ldlm.namespaces.lustre-OST0000-osc-ffffa026923be000.early_lock_cancel=1 ldlm.namespaces.lustre-OST0001-osc-ffffa026923be000.early_lock_cancel=1 Connected clients: oleg145-client.virtnet oleg145-client.virtnet /mnt/lustre has perms 0777 OK PASS 10b (27s) == recovery-small test 10c: re-send BL AST vs reconnect race (LU-5569) ========================================================== 14:40:15 (1788201615) ldlm.namespaces.MGC192.168.201.145@tcp.early_lock_cancel=0 ldlm.namespaces.lustre-MDT0000-mdc-ffffa026923be000.early_lock_cancel=0 ldlm.namespaces.lustre-OST0000-osc-ffffa026923be000.early_lock_cancel=0 ldlm.namespaces.lustre-OST0001-osc-ffffa026923be000.early_lock_cancel=0 fail_loc=0x80000305 mdc.lustre-MDT0000-mdc-ffffa026923be000.import=connection=192.168.201.145@tcp fail_loc=0 fail_val=0 ldlm.namespaces.MGC192.168.201.145@tcp.early_lock_cancel=1 ldlm.namespaces.lustre-MDT0000-mdc-ffffa026923be000.early_lock_cancel=1 ldlm.namespaces.lustre-OST0000-osc-ffffa026923be000.early_lock_cancel=1 ldlm.namespaces.lustre-OST0001-osc-ffffa026923be000.early_lock_cancel=1 Connected clients: oleg145-client.virtnet oleg145-client.virtnet /mnt/lustre/d10c.recovery-small has perms 0777 OK PASS 10c (11s) == recovery-small test 10d: test failed blocking ast ===== 14:40:26 (1788201626) 7+0 records in 7+0 records out 7 bytes copied, 0.020866 s, 0.3 kB/s Stopping client oleg145-client.virtnet /mnt/lustre (opts:) Starting client: oleg145-client.virtnet: -o user_xattr,flock 192.168.201.145@tcp:/lustre /mnt/lustre Starting client: oleg145-client.virtnet: -o user_xattr,flock 192.168.201.145@tcp:/lustre /mnt/lustre2 1+0 records in 1+0 records out 5 bytes copied, 0.0113074 s, 0.4 kB/s fail_err=71 ldlm.namespaces.MGC192.168.201.145@tcp.early_lock_cancel=0 ldlm.namespaces.lustre-MDT0000-mdc-ffffa02682aaf800.early_lock_cancel=0 ldlm.namespaces.lustre-MDT0000-mdc-ffffa02682d4d000.early_lock_cancel=0 ldlm.namespaces.lustre-OST0000-osc-ffffa02682aaf800.early_lock_cancel=0 ldlm.namespaces.lustre-OST0000-osc-ffffa02682d4d000.early_lock_cancel=0 ldlm.namespaces.lustre-OST0001-osc-ffffa02682aaf800.early_lock_cancel=0 ldlm.namespaces.lustre-OST0001-osc-ffffa02682d4d000.early_lock_cancel=0 fail_loc=0x305 fail_loc=0 fail_val=0 ldlm.namespaces.MGC192.168.201.145@tcp.early_lock_cancel=1 ldlm.namespaces.lustre-MDT0000-mdc-ffffa02682aaf800.early_lock_cancel=1 ldlm.namespaces.lustre-MDT0000-mdc-ffffa02682d4d000.early_lock_cancel=1 ldlm.namespaces.lustre-OST0000-osc-ffffa02682aaf800.early_lock_cancel=1 ldlm.namespaces.lustre-OST0000-osc-ffffa02682d4d000.early_lock_cancel=1 ldlm.namespaces.lustre-OST0001-osc-ffffa02682aaf800.early_lock_cancel=1 ldlm.namespaces.lustre-OST0001-osc-ffffa02682d4d000.early_lock_cancel=1 uname: write error: Input/output error Connected clients: oleg145-client.virtnet fail_err=0 192.168.201.145@tcp:/lustre /mnt/lustre2 lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg145-client.virtnet /mnt/lustre2 (opts:) PASS 10d (18s) == recovery-small test 10e: re-send BL AST vs reconnect race 2 ========================================================== 14:40:44 (1788201644) SKIP: recovery-small test_10e need two clients SKIP 10e (4s) == recovery-small test 11: wake up a thread waiting for completion after eviction (b=2460) ========================================================== 14:40:48 (1788201648) ldlm.namespaces.MGC192.168.201.145@tcp.early_lock_cancel=0 ldlm.namespaces.lustre-MDT0000-mdc-ffffa02682aaf800.early_lock_cancel=0 ldlm.namespaces.lustre-OST0000-osc-ffffa02682aaf800.early_lock_cancel=0 ldlm.namespaces.lustre-OST0001-osc-ffffa02682aaf800.early_lock_cancel=0 fail_loc=0x80000305 fail_loc=0 fail_val=0 ldlm.namespaces.MGC192.168.201.145@tcp.early_lock_cancel=1 ldlm.namespaces.lustre-MDT0000-mdc-ffffa02682aaf800.early_lock_cancel=1 ldlm.namespaces.lustre-OST0000-osc-ffffa02682aaf800.early_lock_cancel=1 ldlm.namespaces.lustre-OST0001-osc-ffffa02682aaf800.early_lock_cancel=1 PASS 11 (25s) == recovery-small test 12: recover from timed out resend in ptlrpcd (b=2494) ========================================================== 14:41:13 (1788201673) fail_loc=0x115 multiop /mnt/lustre/f12.recovery-small vOS_c TMPPIPE=/tmp/multiop_open_wait_pipe.7622 waiting for multiop 17641 clearing fail_loc on mds1 oleg145-server: ssh: connect to host oleg145-server port 22: Connection timed out pdsh@oleg145-client: oleg145-server: ssh exited with exit code 255 PASS 12 (53s) == recovery-small test 13: mdc_readpage restart test (bug 1138) ========================================================== 14:42:06 (1788201726)