[ 1498.512772] LustreError: lustre-OST0000-osc-ffff88012fa26000: This client was evicted by lustre-OST0000; in progress operations using this service will fail.
[ 1498.534865] Lustre: 1970:0:(llite_lib.c:4117:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.202.134@tcp:/lustre/fid: [0x200000404:0x35:0x0]// may get corrupted (rc -108)
[ 1501.988914] Lustre: DEBUG MARKER: == recovery-small test 26a: evict dead exports =========== 17:26:28 (1746134788)
[ 1502.868550] Lustre: DEBUG MARKER: SKIP: recovery-small test_26a msg and ost1 are at the same node
[ 1503.683281] Lustre: DEBUG MARKER: == recovery-small test 26b: evict dead exports =========== 17:26:30 (1746134790)
[ 1504.385458] Lustre: DEBUG MARKER: SKIP: recovery-small test_26b msg and ost1 are at the same node
[ 1505.269119] Lustre: DEBUG MARKER: == recovery-small test 27: fail LOV while using OSC's ==== 17:26:32 (1746134792)
[ 1525.988047] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 192.168.202.134@tcp) was lost; in progress operations using this service will fail
[ 1526.016507] Lustre: Evicted from MGS (at 192.168.202.134@tcp) after server handle changed from 0x46b199b7805bf3f to 0x46b199b7805e104
[ 1618.090011] LustreError: lustre-MDT0000-mdc-ffff88012fa26000: operation mds_reint to node 192.168.202.134@tcp failed: rc = -19
[ 1618.092982] LustreError: Skipped 2 previous similar messages
[ 1636.125262] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 192.168.202.134@tcp) was lost; in progress operations using this service will fail
[ 1636.154001] Lustre: Evicted from MGS (at 192.168.202.134@tcp) after server handle changed from 0x46b199b7805e104 to 0x46b199b780a3f21
[ 1640.132098] Lustre: DEBUG MARKER: == recovery-small test 28: handle error adding new clients (bug 6086) ========================================================== 17:28:46 (1746134926)
[ 1640.446695] Lustre: *** cfs_fail_loc=305, val=0***
[ 1640.449415] Lustre: Skipped 4 previous similar messages
[ 1658.370601] LustreError: lustre-OST0001-osc-ffff88012fa26000: operation ost_connect to node 192.168.202.134@tcp failed: rc = -75
[ 1681.183587] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 192.168.202.134@tcp) was lost; in progress operations using this service will fail
[ 1681.204218] Lustre: Evicted from MGS (at 192.168.202.134@tcp) after server handle changed from 0x46b199b780a3f21 to 0x46b199b780a4445
[ 1683.771287] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 1684.622583] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 1689.442137] Lustre: DEBUG MARKER: == recovery-small test 29a: error adding new clients doesn't cause LBUG (bug 22273) ========================================================== 17:29:36 (1746134976)
[ 1696.205620] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 192.168.202.134@tcp) was lost; in progress operations using this service will fail
[ 1696.222573] Lustre: Evicted from MGS (at 192.168.202.134@tcp) after server handle changed from 0x46b199b780a4445 to 0x46b199b780a483c
[ 1696.272003] LustreError: lustre-MDT0000-mdc-ffff88012fa26000: This client was evicted by lustre-MDT0000; in progress operations using this service will fail.
[ 1709.026395] Lustre: DEBUG MARKER: == recovery-small test 29b: error adding new clients doesn't cause LBUG (bug 22273) ========================================================== 17:29:55 (1746134995)
[ 1717.039426] LustreError: lustre-OST0000-osc-ffff88012fa26000: This client was evicted by lustre-OST0000; in progress operations using this service will fail.
[ 1725.649599] Lustre: DEBUG MARKER: == recovery-small test 50: failover MDS under load ======= 17:30:12 (1746135012)
[ 1736.897152] LustreError: lustre-MDT0000-mdc-ffff88012fa26000: operation ldlm_enqueue to node 192.168.202.134@tcp failed: rc = -19
[ 1756.302465] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 192.168.202.134@tcp) was lost; in progress operations using this service will fail
[ 1756.319839] Lustre: Evicted from MGS (at 192.168.202.134@tcp) after server handle changed from 0x46b199b780a483c to 0x46b199b780aec5a
[ 1763.107116] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 1764.136253] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 1842.474091] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 192.168.202.134@tcp) was lost; in progress operations using this service will fail
[ 1842.505167] Lustre: Evicted from MGS (at 192.168.202.134@tcp) after server handle changed from 0x46b199b780aec5a to 0x46b199b780de4e4
[ 1850.148770] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 1851.287432] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 1914.100855] LustreError: lustre-MDT0000-mdc-ffff88012fa26000: operation ldlm_enqueue to node 192.168.202.134@tcp failed: rc = -19
[ 1914.111187] LustreError: Skipped 3 previous similar messages
[ 1932.656138] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 192.168.202.134@tcp) was lost; in progress operations using this service will fail
[ 1932.682734] Lustre: Evicted from MGS (at 192.168.202.134@tcp) after server handle changed from 0x46b199b780de4e4 to 0x46b199b7810ed0b
[ 1935.715890] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 1936.847419] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 1962.313829] Lustre: DEBUG MARKER: == recovery-small test 51: failover MDS during recovery == 17:34:09 (1746135249)
[ 1982.736542] Lustre: MGC192.168.202.134@tcp: Connection restored to 192.168.202.134@tcp (at 192.168.202.134@tcp)
[ 1982.745487] Lustre: Skipped 22 previous similar messages
[ 1984.327007] Lustre: DEBUG MARKER: test_51: failover in 1 sec
[ 1986.629030] Lustre: lustre-MDT0000-mdc-ffff88012fa26000: Connection to lustre-MDT0000 (at 192.168.202.134@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 1986.645068] Lustre: Skipped 16 previous similar messages
[ 2002.735581] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 192.168.202.134@tcp) was lost; in progress operations using this service will fail
[ 2002.749587] LustreError: Skipped 1 previous similar message
[ 2002.808424] Lustre: Evicted from MGS (at 192.168.202.134@tcp) after server handle changed from 0x46b199b781236e4 to 0x46b199b781266be
[ 2002.812310] Lustre: Skipped 1 previous similar message
[ 2007.739515] Lustre: DEBUG MARKER: test_51: failover in 5 sec
[ 2032.334425] Lustre: DEBUG MARKER: test_51: failover in 10 sec
[ 2065.563005] Lustre: DEBUG MARKER: test_51: failover in 20 sec
[ 2106.985507] Lustre: DEBUG MARKER: test_51: failover in 25 sec
[ 2147.961619] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 192.168.202.134@tcp) was lost; in progress operations using this service will fail
[ 2147.969198] LustreError: Skipped 3 previous similar messages
[ 2147.987398] Lustre: Evicted from MGS (at 192.168.202.134@tcp) after server handle changed from 0x46b199b78140975 to 0x46b199b781539bd
[ 2147.992934] Lustre: Skipped 3 previous similar messages
[ 2151.376141] Lustre: DEBUG MARKER: test_51: failover in 30 sec
[ 2182.665804] LustreError: lustre-MDT0000-mdc-ffff88012fa26000: operation mds_reint to node 192.168.202.134@tcp failed: rc = -19
[ 2182.678451] LustreError: Skipped 9 previous similar messages
[ 2226.150323] Lustre: DEBUG MARKER: == recovery-small test 52: failover OST under load ======= 17:38:33 (1746135513)
[ 2260.779148] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid
[ 2261.674136] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec
[ 2590.663026] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid
[ 2591.670769] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec
[ 2899.635540] LustreError: lustre-OST0000-osc-ffff88012fa26000: operation ldlm_enqueue to node 192.168.202.134@tcp failed: rc = -19
[ 2899.639862] LustreError: Skipped 3 previous similar messages
[ 2899.641542] Lustre: lustre-OST0000-osc-ffff88012fa26000: Connection to lustre-OST0000 (at 192.168.202.134@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 2899.648028] Lustre: Skipped 7 previous similar messages
[ 2920.452186] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid
[ 2921.177066] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec
[ 3194.077412] Lustre: DEBUG MARKER: == recovery-small test 53a: touch: drop rep ============== 17:54:40 (1746136480)
[ 3211.402303] Lustre: 22388:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1746136483/real 1746136483]  req@ffff88012e2a2300 x1830953555273344/t0(0) o101->lustre-MDT0000-mdc-ffff88012fa26000@192.168.202.134@tcp:12/10 lens 576/1024 e 0 to 1 dl 1746136499 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'openfile.0' uid:0 gid:0
[ 3211.420680] Lustre: 22388:0:(client.c:2346:ptlrpc_expire_one_request()) Skipped 7 previous similar messages
[ 3211.462665] Lustre: lustre-MDT0000-mdc-ffff88012fa26000: Connection restored to 192.168.202.134@tcp (at 192.168.202.134@tcp)
[ 3211.465983] Lustre: Skipped 14 previous similar messages
[ 3215.617685] Lustre: DEBUG MARKER: == recovery-small test 53b: touch: drop rep ============== 17:55:02 (1746136502)
[ 3235.985148] Lustre: DEBUG MARKER: == recovery-small test 53c: touch: drop rep ============== 17:55:22 (1746136522)
[ 3256.723719] Lustre: DEBUG MARKER: == recovery-small test 54: back in time ================== 17:55:43 (1746136543)
[ 3256.974233] Lustre: Mounted lustre-client
[ 3282.014070] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 192.168.202.134@tcp) was lost; in progress operations using this service will fail
[ 3282.020661] LustreError: Skipped 1 previous similar message
[ 3282.030647] Lustre: Evicted from MGS (at 192.168.202.134@tcp) after server handle changed from 0x46b199b78166d23 to 0x46b199b7848e73f
[ 3282.039028] Lustre: Skipped 1 previous similar message
[ 3288.019659] Lustre: 1967:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1746136559/real 1746136559]  req@ffff8800a87a3800 x1830953555289216/t0(0) o400->lustre-MDT0000-mdc-ffff88012fa26000@192.168.202.134@tcp:12/10 lens 224/224 e 0 to 1 dl 1746136575 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[ 3288.038161] Lustre: 1967:0:(client.c:2346:ptlrpc_expire_one_request()) Skipped 2 previous similar messages
[ 3288.301272] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 3289.135227] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 3290.741030] LustreError: 25393:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup
[ 3290.743914] LustreError: 25393:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages
[ 3290.789412] Lustre: Unmounted lustre-client
[ 3293.654359] Lustre: DEBUG MARKER: == recovery-small test 55: ost_brw_read/write drops timed-out read/write request ========================================================== 17:56:20 (1746136580)
[ 3332.085783] Lustre: lustre-OST0000-osc-ffff88012fa26000: Connection restored to 192.168.202.134@tcp (at 192.168.202.134@tcp)
[ 3332.090939] Lustre: Skipped 5 previous similar messages
[ 3444.160412] Lustre: 1968:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1746136715/real 1746136715]  req@ffff8800a87a3800 x1830953555337344/t0(0) o4->lustre-OST0000-osc-ffff88012fa26000@192.168.202.134@tcp:6/4 lens 488/448 e 0 to 1 dl 1746136731 ref 2 fl Rpc:XQr/202/ffffffff rc -11/-1 job:'dd.0' uid:0 gid:0
[ 3444.180624] Lustre: 1968:0:(client.c:2346:ptlrpc_expire_one_request()) Skipped 74 previous similar messages
[ 3492.217111] Lustre: lustre-OST0000-osc-ffff88012fa26000: Connection restored to 192.168.202.134@tcp (at 192.168.202.134@tcp)
[ 3492.223662] Lustre: Skipped 9 previous similar messages
[ 3508.229379] Lustre: lustre-OST0000-osc-ffff88012fa26000: Connection to lustre-OST0000 (at 192.168.202.134@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 3508.240924] Lustre: Skipped 17 previous similar messages
[ 3581.099321] Lustre: DEBUG MARKER: == recovery-small test 56: do not fail on getattr resend ========================================================== 18:01:08 (1746136868)
[ 3624.979945] Lustre: DEBUG MARKER: == recovery-small test 57: read procfs entries causes kernel crash ========================================================== 18:01:51 (1746136911)
[ 3626.493277] LustreError: 27940:0:(obd_class.h:479:obd_check_dev()) Device 5 not setup
[ 3626.496344] LustreError: 27940:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages
[ 3626.518064] Lustre: Unmounted lustre-client
[ 3634.696869] Lustre: Mounted lustre-client
[ 3636.314010] Lustre: DEBUG MARKER: == recovery-small test 58: Eviction in the middle of open RPC reply processing ========================================================== 18:02:03 (1746136923)
[ 3636.388383] LustreError: 28991:0:(mdc_locks.c:1333:mdc_finish_intent_lock()) cfs_fail_timeout id 801 sleeping for 20000ms
[ 3637.391589] LustreError: 28991:0:(mdc_locks.c:1333:mdc_finish_intent_lock()) cfs_fail_timeout interrupted
[ 3637.436594] Lustre: *** cfs_fail_loc=305, val=0***
[ 3655.346994] Lustre: DEBUG MARKER: == recovery-small test 59: Read cancel race on client eviction ========================================================== 18:02:22 (1746136942)
[ 3655.499168] Lustre: Mounted lustre-client
[ 3655.763409] Lustre: setting import lustre-MDT0000_UUID INACTIVE by administrator request
[ 3665.790485] LustreError: 29732:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup
[ 3665.793892] LustreError: 29732:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages
[ 3665.804172] Lustre: Unmounted lustre-client
[ 3667.630159] Lustre: DEBUG MARKER: == recovery-small test 60: Add Changelog entries during MDS failover ========================================================== 18:02:34 (1746136954)
[ 3702.482364] LustreError: lustre-MDT0000-mdc-ffff88009ed3b800: operation mds_reint to node 192.168.202.134@tcp failed: rc = -19
[ 3715.593395] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 192.168.202.134@tcp) was lost; in progress operations using this service will fail
[ 3715.611661] Lustre: Evicted from MGS (at 192.168.202.134@tcp) after server handle changed from 0x46b199b7848f3cc to 0x46b199b784bbc2f
[ 3778.102861] Lustre: DEBUG MARKER: == recovery-small test 61: Verify to not reuse orphan objects - bug 17025 ========================================================== 18:04:25 (1746137065)
[ 3779.946344] Lustre: DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000
[ 3785.720500] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 192.168.202.134@tcp) was lost; in progress operations using this service will fail
[ 3785.731041] Lustre: Evicted from MGS (at 192.168.202.134@tcp) after server handle changed from 0x46b199b784bbc2f to 0x46b199b78505f8f
[ 3785.756972] LustreError: lustre-MDT0000-mdc-ffff88009ed3b800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail.
[ 3791.688009] Lustre: DEBUG MARKER: == recovery-small test 65: lock enqueue for destroyed export ========================================================== 18:04:38 (1746137078)
[ 3791.806673] Lustre: Mounted lustre-client
[ 3796.817916] LustreError: lustre-OST0000-osc-ffff88009ed3b800: This client was evicted by lustre-OST0000; in progress operations using this service will fail.
[ 3808.823653] Lustre: 1968:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1746137080/real 1746137080]  req@ffff8800a855bb80 x1830953559984640/t0(0) o103->lustre-OST0000-osc-ffff8800a7e3f800@192.168.202.134@tcp:17/18 lens 328/224 e 0 to 1 dl 1746137096 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'ldlm_bl.0' uid:0 gid:0
[ 3808.840170] Lustre: 1968:0:(client.c:2346:ptlrpc_expire_one_request()) Skipped 71 previous similar messages
[ 3808.878153] Lustre: lustre-OST0000-osc-ffff8800a7e3f800: Connection restored to  (at 192.168.202.134@tcp)
[ 3808.893025] Lustre: Skipped 11 previous similar messages
[ 3809.622234] LustreError: 576:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup
[ 3809.626371] LustreError: 576:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages
[ 3809.660802] Lustre: Unmounted lustre-client
[ 3812.621875] Lustre: DEBUG MARKER: == recovery-small test 66: lock enqueue re-send vs client eviction ========================================================== 18:04:59 (1746137099)
[ 3817.562470] LustreError: lustre-MDT0000-mdc-ffff88009ed3b800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail.
[ 3817.570919] LustreError: 1311:0:(file.c:6135:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -5
[ 3820.563890] Lustre: DEBUG MARKER: == recovery-small test 67: connect vs import invalidate race ========================================================== 18:05:07 (1746137107)
[ 3820.669236] LustreError: 2093:0:(recover.c:322:ptlrpc_recover_import()) cfs_race id 531 sleeping
[ 3825.421109] LustreError: lustre-MDT0000-mdc-ffff88009ed3b800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail.
[ 3825.427945] LustreError: 2113:0:(import.c:287:ptlrpc_invalidate_import()) cfs_fail_race id 531 waking
[ 3825.435356] LustreError: 2093:0:(recover.c:322:ptlrpc_recover_import()) cfs_fail_race id 531 awake: rc=238
[ 3825.440044] LustreError: 2093:0:(import.c:709:ptlrpc_connect_import_locked()) already connecting
[ 3825.451151] LustreError: 2118:0:(file.c:6135:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108
[ 3826.493013] LustreError: 2125:0:(file.c:6135:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108
[ 3826.498988] LustreError: 2125:0:(file.c:6135:ll_inode_revalidate_fini()) Skipped 5 previous similar messages
[ 3828.612186] LustreError: 2149:0:(file.c:6135:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108
[ 3828.618846] LustreError: 2149:0:(file.c:6135:ll_inode_revalidate_fini()) Skipped 13 previous similar messages
[ 3832.890484] LustreError: 2196:0:(file.c:6135:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108
[ 3832.901116] LustreError: 2196:0:(file.c:6135:ll_inode_revalidate_fini()) Skipped 27 previous similar messages
[ 3839.818342] Lustre: DEBUG MARKER: == recovery-small test 100: IR: Make sure normal recovery still works w/o IR ========================================================== 18:05:26 (1746137126)
[ 3861.905391] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid
[ 3862.482706] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec
[ 3866.369127] Lustre: DEBUG MARKER: == recovery-small test 101a: IR: Make sure IR works w/o normal recovery ========================================================== 18:05:53 (1746137153)
[ 3885.070561] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid
[ 3885.663837] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec
[ 3889.717751] Lustre: DEBUG MARKER: == recovery-small test 101b: IR: Make sure IR works w/o normal recovery and proceed EAGAIN ========================================================== 18:06:16 (1746137176)
[ 3935.124058] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid
[ 3935.737032] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec
[ 3939.292350] Lustre: DEBUG MARKER: == recovery-small test 102: IR: New client gets updated nidtbl after MGS restart ========================================================== 18:07:06 (1746137226)
[ 3960.833087] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid
[ 3961.536092] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec
[ 3964.169333] LustreError: 8236:0:(obd_class.h:479:obd_check_dev()) Device 5 not setup
[ 3964.174356] LustreError: 8236:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages
[ 3964.193623] Lustre: Unmounted lustre-client
[ 3989.363327] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid
[ 3990.958390] Lustre: Mounted lustre-client
[ 3993.496218] Lustre: DEBUG MARKER: == recovery-small test 103: IR: MDS can start w/o MGS and get updated nidtbl later ========================================================== 18:08:00 (1746137280)
[ 3994.441741] Lustre: DEBUG MARKER: SKIP: recovery-small test_103 needs separate mgs and mds
[ 3995.353462] Lustre: DEBUG MARKER: == recovery-small test 104: IR: ost can disable IR voluntarily ========================================================== 18:08:02 (1746137282)
[ 4006.760050] Lustre: DEBUG MARKER: == recovery-small test 105: IR: NON IR clients support === 18:08:13 (1746137293)
[ 4007.392884] Lustre: DEBUG MARKER: SKIP: recovery-small test_105 Needs multiple clients
[ 4008.074166] Lustre: DEBUG MARKER: == recovery-small test 106: lightweight connection support ========================================================== 18:08:15 (1746137295)
[ 4008.687799] Lustre: *** cfs_fail_loc=805, val=0***
[ 4008.705651] Lustre: Mounted lustre-client
[ 4010.676845] Lustre: DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000
[ 4027.301426] LustreError: 1966:0:(ldlm_resource.c:982:ldlm_resource_complain()) lustre-MDT0000-mdc-ffff8800b3e2d000: namespace resource [0x200000007:0x1:0x0].0x0 (ffff88009ecbbb00) refcount nonzero (1) after lock cleanup; forcing cleanup.
[ 4027.998491] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 192.168.202.134@tcp) was lost; in progress operations using this service will fail
[ 4028.002700] LustreError: 11756:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup
[ 4028.002702] LustreError: 11756:0:(obd_class.h:479:obd_check_dev()) Skipped 7 previous similar messages
[ 4028.021773] Lustre: Evicted from MGS (at 192.168.202.134@tcp) after server handle changed from 0x46b199b78506959 to 0x46b199b78506f3a
[ 4028.024881] Lustre: Unmounted lustre-client
[ 4030.878775] Lustre: DEBUG MARKER: == recovery-small test 107: drop reint reply, then restart MDT ========================================================== 18:08:38 (1746137318)
[ 4047.436887] LustreError: 1966:0:(client.c:3262:ptlrpc_replay_interpret()) @@@ status 301, old was 0  req@ffff8800a855ad80 x1830953560051328/t94489280515(94489280515) o101->lustre-MDT0000-mdc-ffff88009ede0800@192.168.202.134@tcp:12/10 lens 664/608 e 0 to 0 dl 1746137351 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'multiop.0' uid:0 gid:0
[ 4050.941499] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 4051.502022] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 4052.864479] checkstat (13260) used greatest stack depth: 10560 bytes left
[ 4055.440613] Lustre: DEBUG MARKER: == recovery-small test 108: client eviction don't crash == 18:09:02 (1746137342)
[ 4056.109328] LustreError: lustre-OST0000-osc-ffff88009ede0800: This client was evicted by lustre-OST0000; in progress operations using this service will fail.
[ 4056.122990] Lustre: 1968:0:(llite_lib.c:4117:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.202.134@tcp:/lustre/fid: [0x20000a042:0x6:0x0]// may get corrupted (rc -5)
[ 4056.158081] LustreError: 13920:0:(ldlm_resource.c:982:ldlm_resource_complain()) lustre-OST0000-osc-ffff88009ede0800: namespace resource [0x240000400:0x7d42:0x0].0x0 (ffff88009ecbba00) refcount nonzero (1) after lock cleanup; forcing cleanup.
[ 4056.174944] LustreError: 13920:0:(ldlm_resource.c:982:ldlm_resource_complain()) Skipped 1 previous similar message
[ 4059.758282] Lustre: DEBUG MARKER: == recovery-small test 110a: create remote directory: drop client req ========================================================== 18:09:06 (1746137346)
[ 4060.434261] Lustre: DEBUG MARKER: SKIP: recovery-small test_110a needs >= 2 MDTs
[ 4061.128163] Lustre: DEBUG MARKER: == recovery-small test 110b: create remote directory: drop Master rep ========================================================== 18:09:08 (1746137348)
[ 4061.696415] Lustre: DEBUG MARKER: SKIP: recovery-small test_110b needs >= 2 MDTs
[ 4062.347964] Lustre: DEBUG MARKER: == recovery-small test 110c: create remote directory: drop update rep on slave MDT ========================================================== 18:09:09 (1746137349)
[ 4062.968868] Lustre: DEBUG MARKER: SKIP: recovery-small test_110c needs >= 2 MDTs
[ 4063.597625] Lustre: DEBUG MARKER: == recovery-small test 110d: remove remote directory: drop client req ========================================================== 18:09:10 (1746137350)
[ 4064.129107] Lustre: DEBUG MARKER: SKIP: recovery-small test_110d needs >= 2 MDTs
[ 4064.730412] Lustre: DEBUG MARKER: == recovery-small test 110e: remove remote directory: drop master rep ========================================================== 18:09:11 (1746137351)
[ 4065.338202] Lustre: DEBUG MARKER: SKIP: recovery-small test_110e needs >= 2 MDTs
[ 4065.998293] Lustre: DEBUG MARKER: == recovery-small test 110f: remove remote directory: drop slave rep ========================================================== 18:09:13 (1746137353)
[ 4066.594601] Lustre: DEBUG MARKER: SKIP: recovery-small test_110f needs >= 2 MDTs
[ 4067.229499] Lustre: DEBUG MARKER: == recovery-small test 110g: drop reply during migration ========================================================== 18:09:14 (1746137354)
[ 4067.787049] Lustre: DEBUG MARKER: SKIP: recovery-small test_110g needs >= 2 MDTs
[ 4068.449260] Lustre: DEBUG MARKER: == recovery-small test 110h: drop update reply during cross-MDT file rename ========================================================== 18:09:15 (1746137355)
[ 4068.997304] Lustre: DEBUG MARKER: SKIP: recovery-small test_110h needs >= 2 MDTs
[ 4069.643021] Lustre: DEBUG MARKER: == recovery-small test 110i: drop update reply during cross-MDT dir rename ========================================================== 18:09:16 (1746137356)
[ 4070.207972] Lustre: DEBUG MARKER: SKIP: recovery-small test_110i needs >= 2 MDTs
[ 4070.964628] Lustre: DEBUG MARKER: == recovery-small test 110j: drop update reply during cross-MDT ln ========================================================== 18:09:18 (1746137358)
[ 4071.567986] Lustre: DEBUG MARKER: SKIP: recovery-small test_110j needs >= 2 MDTs
[ 4072.274443] Lustre: DEBUG MARKER: == recovery-small test 110k: FID_QUERY failed during recovery ========================================================== 18:09:19 (1746137359)
[ 4072.864845] Lustre: DEBUG MARKER: SKIP: recovery-small test_110k needs >= 2 MDTS
[ 4073.764554] Lustre: DEBUG MARKER: == recovery-small test 110m: update resent vs original RPC race ========================================================== 18:09:20 (1746137360)
[ 4074.878779] Lustre: DEBUG MARKER: SKIP: recovery-small test_110m needs at least 2 MDTs
[ 4075.623833] Lustre: DEBUG MARKER: == recovery-small test 111: mdd setup fail should not cause umount oops ========================================================== 18:09:22 (1746137362)
[ 4087.970192] Lustre: DEBUG MARKER: == recovery-small test 112a: bulk resend while orignal request is in progress ========================================================== 18:09:35 (1746137375)
[ 4111.362759] Lustre: DEBUG MARKER: == recovery-small test 115a: read: late REQ MDunlink and no bulk ========================================================== 18:09:58 (1746137398)
[ 4111.597007] Lustre: *** cfs_fail_loc=51b, val=3***
[ 4118.118958] Lustre: DEBUG MARKER: == recovery-small test 115b: write: late REQ MDunlink and no bulk ========================================================== 18:10:05 (1746137405)
[ 4118.264739] Lustre: *** cfs_fail_loc=51b, val=4***
[ 4124.807213] Lustre: DEBUG MARKER: == recovery-small test 115c: read: late Reply MDunlink and no bulk ========================================================== 18:10:11 (1746137411)
[ 4124.988200] Lustre: *** cfs_fail_loc=50f, val=3***
[ 4129.041221] Lustre: DEBUG MARKER: == recovery-small test 115d: write: late Reply MDunlink and no bulk ========================================================== 18:10:16 (1746137416)
[ 4129.170678] Lustre: *** cfs_fail_loc=50f, val=4***
[ 4132.675933] Lustre: DEBUG MARKER: == recovery-small test 115e: read: late Bulk MDunlink and no reply ========================================================== 18:10:19 (1746137419)
[ 4132.809622] Lustre: *** cfs_fail_loc=510, val=3***
[ 4136.706657] Lustre: DEBUG MARKER: == recovery-small test 115f: read: late REQ MDunlink and no reply ========================================================== 18:10:23 (1746137423)
[ 4136.867087] Lustre: *** cfs_fail_loc=51b, val=3***
[ 4143.680848] Lustre: DEBUG MARKER: == recovery-small test 115g: read: late REQ MDunlink and Reply MDunlink ========================================================== 18:10:30 (1746137430)
[ 4202.865127] Lustre: lustre-OST0001-osc-ffff88009ede0800: Connection to lustre-OST0001 (at 192.168.202.134@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 4202.873186] Lustre: Skipped 24 previous similar messages
[ 4205.123183] Lustre: DEBUG MARKER: == recovery-small test 120: flock race: completion vs. evict ========================================================== 18:11:32 (1746137492)
[ 4205.339662] LustreError: 25082:0:(ldlm_flock.c:802:ldlm_flock_completion_ast()) cfs_fail_timeout id 320 sleeping for 4000ms
[ 4207.721241] LustreError: lustre-MDT0000-mdc-ffff88009ede0800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail.
[ 4207.732391] LustreError: 25098:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff88009ede0800: inode [0x20000a042:0x8:0x0] mdc close failed: rc = -108
[ 4207.750148] LustreError: 25098:0:(ldlm_resource.c:982:ldlm_resource_complain()) lustre-MDT0000-mdc-ffff88009ede0800: namespace resource [0x20000a042:0x11:0x0].0xc (ffff88012ab7b400) refcount nonzero (1) after lock cleanup; forcing cleanup.
[ 4207.759460] LustreError: 25103:0:(file.c:6135:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108
[ 4207.764190] LustreError: 25103:0:(file.c:6135:ll_inode_revalidate_fini()) Skipped 20 previous similar messages
[ 4209.348807] LustreError: 25082:0:(ldlm_flock.c:802:ldlm_flock_completion_ast()) cfs_fail_timeout id 320 awake
[ 4211.389243] LustreError: 25116:0:(ldlm_flock.c:802:ldlm_flock_completion_ast()) cfs_fail_timeout id 320 sleeping for 4000ms
[ 4213.720787] LustreError: lustre-MDT0000-mdc-ffff88009ede0800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail.
[ 4213.730792] LustreError: 25132:0:(ldlm_resource.c:982:ldlm_resource_complain()) lustre-MDT0000-mdc-ffff88009ede0800: namespace resource [0x20000a042:0x11:0x0].0xc (ffff8800b6f74800) refcount nonzero (1) after lock cleanup; forcing cleanup.
[ 4213.739545] LustreError: 25132:0:(ldlm_resource.c:982:ldlm_resource_complain()) Skipped 1 previous similar message
[ 4215.393640] LustreError: 25116:0:(ldlm_flock.c:802:ldlm_flock_completion_ast()) cfs_fail_timeout id 320 awake
[ 4221.755359] LustreError: lustre-MDT0000-mdc-ffff88009ede0800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail.
[ 4221.882570] LustreError: 25179:0:(ldlm_flock.c:807:ldlm_flock_completion_ast()) cfs_fail_timeout id 321 sleeping for 4000ms
[ 4224.234868] LustreError: lustre-MDT0000-mdc-ffff88009ede0800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail.
[ 4224.261319] LustreError: 25200:0:(file.c:6135:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108
[ 4225.887678] LustreError: 25179:0:(ldlm_flock.c:807:ldlm_flock_completion_ast()) cfs_fail_timeout id 321 awake
[ 4225.891953] LustreError: 25179:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff88009ede0800: inode [0x20000a042:0x11:0x0] mdc close failed: rc = -108
[ 4225.897116] LustreError: 25179:0:(file.c:247:ll_close_inode_openhandle()) Skipped 4 previous similar messages
[ 4230.701559] LustreError: 25259:0:(ldlm_flock.c:807:ldlm_flock_completion_ast()) cfs_fail_timeout id 321 sleeping for 4000ms
[ 4233.115978] LustreError: lustre-MDT0000-mdc-ffff88009ede0800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail.
[ 4233.128004] LustreError: 25275:0:(ldlm_resource.c:982:ldlm_resource_complain()) lustre-MDT0000-mdc-ffff88009ede0800: namespace resource [0x20000a042:0x11:0x0].0xc (ffff8800aba4be00) refcount nonzero (1) after lock cleanup; forcing cleanup.
[ 4233.136833] LustreError: 25275:0:(ldlm_resource.c:982:ldlm_resource_complain()) Skipped 1 previous similar message
[ 4234.711035] LustreError: 25259:0:(ldlm_flock.c:807:ldlm_flock_completion_ast()) cfs_fail_timeout id 321 awake
[ 4241.081069] LustreError: lustre-MDT0000-mdc-ffff88009ede0800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail.
[ 4241.253810] LustreError: 25321:0:(ldlm_flock.c:856:ldlm_flock_completion_ast()) cfs_fail_timeout id 322 sleeping for 4000ms
[ 4243.640231] LustreError: lustre-MDT0000-mdc-ffff88009ede0800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail.
[ 4243.651065] LustreError: 25337:0:(ldlm_resource.c:982:ldlm_resource_complain()) lustre-MDT0000-mdc-ffff88009ede0800: namespace resource [0x20000a042:0x11:0x0].0xc (ffff8800aba4b200) refcount nonzero (1) after lock cleanup; forcing cleanup.
[ 4243.662332] LustreError: 25337:0:(ldlm_resource.c:982:ldlm_resource_complain()) Skipped 1 previous similar message
[ 4245.266764] LustreError: 25321:0:(ldlm_flock.c:856:ldlm_flock_completion_ast()) cfs_fail_timeout id 322 awake
[ 4247.723422] LustreError: lustre-MDT0000-mdc-ffff88009ede0800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail.
[ 4249.340094] LustreError: 25351:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff88009ede0800: inode [0x20000a042:0x11:0x0] mdc close failed: rc = -108
[ 4258.485814] LustreError: lustre-MDT0000-mdc-ffff88009ede0800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail.
[ 4258.492835] LustreError: 25448:0:(ldlm_flock.c:802:ldlm_flock_completion_ast()) cfs_fail_timeout id 320 sleeping for 4000ms
[ 4258.499441] LustreError: 25448:0:(ldlm_flock.c:802:ldlm_flock_completion_ast()) Skipped 2 previous similar messages
[ 4258.511920] LustreError: 25453:0:(file.c:6135:ll_inode_revalidate_fini()) lustre: revalidate FID [0x200000007:0x1:0x0] error: rc = -108
[ 4258.516642] LustreError: 25453:0:(file.c:6135:ll_inode_revalidate_fini()) Skipped 53 previous similar messages
[ 4260.177008] LustreError: 25427:0:(ldlm_flock.c:802:ldlm_flock_completion_ast()) cfs_fail_timeout id 320 awake
[ 4260.182228] LustreError: 25427:0:(ldlm_flock.c:802:ldlm_flock_completion_ast()) Skipped 1 previous similar message
[ 4260.190774] LustreError: 25427:0:(file.c:247:ll_close_inode_openhandle()) lustre-clilmv-ffff88009ede0800: inode [0x20000a042:0x11:0x0] mdc close failed: rc = -108
[ 4265.570693] Lustre: DEBUG MARKER: == recovery-small test 113: ldlm enqueue dropped reply should not cause deadlocks ========================================================== 18:12:32 (1746137552)
[ 4286.844635] LustreError: 11757:0:(ldlm_lockd.c:2901:ldlm_bl_thread_blwi()) cfs_fail_timeout id 31f awake
[ 4286.847905] LustreError: 11757:0:(ldlm_lockd.c:2901:ldlm_bl_thread_blwi()) Skipped 1 previous similar message
[ 4289.900260] Lustre: DEBUG MARKER: == recovery-small test 130a: enqueue resend on not existing file ========================================================== 18:12:56 (1746137576)
[ 4334.326415] Lustre: DEBUG MARKER: == recovery-small test 130b: enqueue resend on a stale inode ========================================================== 18:13:41 (1746137621)
[ 4392.168929] Lustre: DEBUG MARKER: == recovery-small test 130c: layout intent resend on a stale inode ========================================================== 18:14:39 (1746137679)
[ 4417.946775] Lustre: DEBUG MARKER: == recovery-small test 132: long punch =================== 18:15:05 (1746137705)
[ 4418.133609] Lustre: Mounted lustre-client
[ 4539.137553] LustreError: 29598:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup
[ 4539.141284] LustreError: 29598:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages
[ 4539.159789] Lustre: Unmounted lustre-client
[ 4542.207474] Lustre: DEBUG MARKER: == recovery-small test 131: IO vs evict results to IO under staled lock ========================================================== 18:17:09 (1746137829)
[ 4542.470111] LustreError: 30270:0:(osc_request.c:2951:osc_build_rpc()) cfs_fail_timeout id 414 sleeping for 4000ms
[ 4542.474380] LustreError: 30270:0:(osc_request.c:2951:osc_build_rpc()) Skipped 1 previous similar message
[ 4544.353034] LustreError: lustre-OST0000-osc-ffff88009ede0800: operation ldlm_enqueue to node 192.168.202.134@tcp failed: rc = -107
[ 4544.359423] LustreError: Skipped 14 previous similar messages
[ 4544.368298] LustreError: lustre-OST0000-osc-ffff88009ede0800: This client was evicted by lustre-OST0000; in progress operations using this service will fail.
[ 4544.380586] LustreError: 30354:0:(ldlm_resource.c:982:ldlm_resource_complain()) lustre-OST0000-osc-ffff88009ede0800: namespace resource [0x240000400:0x7d6d:0x0].0x0 (ffff8800b6f74500) refcount nonzero (1) after lock cleanup; forcing cleanup.
[ 4544.403444] LustreError: 30354:0:(ldlm_resource.c:982:ldlm_resource_complain()) Skipped 1 previous similar message
[ 4546.489107] LustreError: 30270:0:(osc_request.c:2951:osc_build_rpc()) cfs_fail_timeout id 414 awake
[ 4546.493459] Lustre: 1967:0:(llite_lib.c:4117:ll_dirty_page_discard_warn()) lustre: dirty page discard: 192.168.202.134@tcp:/lustre/fid: [0x20000bf89:0x9:0x0]// may get corrupted (rc -108)
[ 4549.110987] Lustre: DEBUG MARKER: == recovery-small test 133: don't fail on flock resend === 18:17:16 (1746137836)
[ 4605.769774] Lustre: 30997:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1746137838/real 1746137838]  req@ffff8800a8735c00 x1830953560217344/t0(0) o101->lustre-MDT0000-mdc-ffff88009ede0800@192.168.202.134@tcp:12/10 lens 328/344 e 0 to 1 dl 1746137893 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'multiop.0' uid:0 gid:0
[ 4605.784638] Lustre: 30997:0:(client.c:2346:ptlrpc_expire_one_request()) Skipped 13 previous similar messages
[ 4605.808415] Lustre: lustre-MDT0000-mdc-ffff88009ede0800: Connection restored to  (at 192.168.202.134@tcp)
[ 4605.811793] Lustre: Skipped 28 previous similar messages
[ 4608.073477] Lustre: DEBUG MARKER: == recovery-small test 134: race between failover and search for reply data free slot ========================================================== 18:18:15 (1746137895)
[ 4608.699773] Lustre: DEBUG MARKER: SKIP: recovery-small test_134 Need 2+ clients, have 1
[ 4609.418490] Lustre: DEBUG MARKER: == recovery-small test 135: DOM: open/create resend to return size ========================================================== 18:18:16 (1746137896)
[ 4668.035237] Lustre: DEBUG MARKER: SKIP: recovery-small test_136 skipping excluded test 136
[ 4668.881799] Lustre: DEBUG MARKER: == recovery-small test 137: late resend must be skipped if already applied ========================================================== 18:19:15 (1746137955)
[ 4725.251991] Lustre: DEBUG MARKER: == recovery-small test 138: Umount MDT during recovery === 18:20:12 (1746138012)
[ 4725.903653] Lustre: DEBUG MARKER: SKIP: recovery-small test_138 needs >= 2 MDTs
[ 4726.610678] Lustre: DEBUG MARKER: == recovery-small test 139: corrupted catid won't cause crash ========================================================== 18:20:13 (1746138013)
[ 4727.236228] Lustre: DEBUG MARKER: SKIP: recovery-small test_139 needs >= 2 MDTs
[ 4727.953057] Lustre: DEBUG MARKER: == recovery-small test 140a: local mount is flagged properly ========================================================== 18:20:15 (1746138015)
[ 4738.880445] Lustre: DEBUG MARKER: == recovery-small test 140b: local mount is excluded from recovery ========================================================== 18:20:25 (1746138025)
[ 4742.889693] Lustre: DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000
[ 4759.609659] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 192.168.202.134@tcp) was lost; in progress operations using this service will fail
[ 4759.618756] LustreError: Skipped 2 previous similar messages
[ 4759.631293] Lustre: Evicted from MGS (at 192.168.202.134@tcp) after server handle changed from 0x46b199b7850774b to 0x46b199b785090b9
[ 4759.639696] Lustre: Skipped 2 previous similar messages
[ 4763.686868] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 4764.214321] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 4768.496438] Lustre: DEBUG MARKER: == recovery-small test 141: do not lose locks on MGS restart ========================================================== 18:20:55 (1746138055)
[ 4769.259027] Lustre: DEBUG MARKER: SKIP: recovery-small test_141 cannot run in local mode or from build tree
[ 4769.960223] Lustre: DEBUG MARKER: == recovery-small test 142: orphan name stub can be cleaned up in startup ========================================================== 18:20:57 (1746138057)
[ 4778.831626] Lustre: DEBUG MARKER: == recovery-small test 143: orphan cleanup thread shouldn't be blocked even delete failed ========================================================== 18:21:05 (1746138065)
[ 4792.273382] Lustre: DEBUG MARKER: == recovery-small test 144a: MDT failover should stop precreation threads ========================================================== 18:21:19 (1746138079)
[ 4816.127847] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid
[ 4817.454012] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec
[ 4897.436587] Lustre: lustre-MDT0000-mdc-ffff88009ede0800: Connection to lustre-MDT0000 (at 192.168.202.134@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 4897.442629] Lustre: Skipped 20 previous similar messages
[ 4898.973026] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 4899.589967] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 4920.561854] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 4921.323232] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 4957.172254] Lustre: DEBUG MARKER: == recovery-small test 144b: orphan cleanup shouldn't be blocked for no objects+failover situation ========================================================== 18:24:03 (1746138243)
[ 4986.206141] Lustre: DEBUG MARKER: oleg234-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid
[ 4987.016919] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec
[ 5141.741787] Lustre: DEBUG MARKER: == recovery-small test 144c: reconnection during orphan cleanup shouldn't lose LAST_ID synchronization ========================================================== 18:27:08 (1746138428)
[ 5213.459588] Lustre: MGC192.168.202.134@tcp: Connection restored to  (at 192.168.202.134@tcp)
[ 5213.462748] Lustre: Skipped 12 previous similar messages
[ 5236.254215] Lustre: DEBUG MARKER: == recovery-small test 145: connect mdtlovs and process update logs after recovery expire ========================================================== 18:28:43 (1746138523)
[ 5236.914151] Lustre: DEBUG MARKER: SKIP: recovery-small test_145 needs >= 3 MDTs
[ 5237.764655] Lustre: DEBUG MARKER: == recovery-small test 146: test eviction is counted properly ========================================================== 18:28:44 (1746138524)
[ 5238.690052] LustreError: lustre-MDT0000-mdc-ffff88009ede0800: operation ldlm_enqueue to node 192.168.202.134@tcp failed: rc = -107
[ 5238.695251] LustreError: Skipped 421 previous similar messages
[ 5238.704133] LustreError: lustre-MDT0000-mdc-ffff88009ede0800: This client was evicted by lustre-MDT0000; in progress operations using this service will fail.
[ 5242.187966] Lustre: DEBUG MARKER: == recovery-small test 147: Check client reconnect ======= 18:28:49 (1746138529)
[ 5407.807931] Lustre: Evicted from lustre-OST0000_UUID (at 192.168.202.134@tcp) after server handle changed from 0x46b199b7860507a to 0x46b199b7860517d
[ 5407.813171] Lustre: Skipped 5 previous similar messages
[ 5407.816168] LustreError: lustre-OST0000-osc-ffff88009ede0800: This client was evicted by lustre-OST0000; in progress operations using this service will fail.
[ 5408.457427] Lustre: DEBUG MARKER: == recovery-small test 148: data corruption through resend ========================================================== 18:31:35 (1746138695)
[ 5429.833704] Lustre: 1969:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1746138697/real 1746138697]  req@ffff88012c7b9c00 x1830953577942016/t0(0) o4->lustre-OST0000-osc-ffff88009ede0800@192.168.202.134@tcp:6/4 lens 4584/448 e 0 to 1 dl 1746138717 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0
[ 5429.846463] Lustre: 1969:0:(client.c:2346:ptlrpc_expire_one_request()) Skipped 11 previous similar messages
[ 5443.106930] Lustre: DEBUG MARKER: == recovery-small test 149: skip orphan removal at umount ========================================================== 18:32:10 (1746138730)
[ 5443.946998] Lustre: DEBUG MARKER: SKIP: recovery-small test_149 needs >= 2 MDTs
[ 5444.768763] Lustre: DEBUG MARKER: == recovery-small test 150: statfs when MDT0 offline with lazystatfs option ========================================================== 18:32:11 (1746138731)
[ 5445.376824] Lustre: DEBUG MARKER: SKIP: recovery-small test_150 needs >= 2 MDTs
[ 5446.151900] Lustre: DEBUG MARKER: == recovery-small test 152: QoS object allocation could be awakened in case of OST failover ========================================================== 18:32:13 (1746138733)
[ 5447.159652] Lustre: DEBUG MARKER: SKIP: recovery-small test_152 MDS Linux kernel does not support killable semaphore
[ 5447.850886] Lustre: DEBUG MARKER: == recovery-small test 153: evict vs reconnect race ====== 18:32:14 (1746138734)
[ 5479.928648] LustreError: MGC192.168.202.134@tcp: Connection to MGS (at 192.168.202.134@tcp) was lost; in progress operations using this service will fail
[ 5479.939437] LustreError: Skipped 5 previous similar messages
[ 5488.224661] Lustre: DEBUG MARKER: == recovery-small test 154a: corruption update llog can be skipped ========================================================== 18:32:55 (1746138775)
[ 5488.835087] Lustre: DEBUG MARKER: SKIP: recovery-small test_154a needs >= 2 MDTs
[ 5489.668070] Lustre: DEBUG MARKER: == recovery-small test 154b: restore update llog after failed recovery ========================================================== 18:32:56 (1746138776)
[ 5490.235805] Lustre: DEBUG MARKER: SKIP: recovery-small test_154b needs >= 2 MDTs
[ 5491.038826] Lustre: DEBUG MARKER: == recovery-small test 155: failover after client remount ========================================================== 18:32:58 (1746138778)
[ 5493.713093] LustreError: 18250:0:(obd_class.h:479:obd_check_dev()) Device 5 not setup
[ 5493.714889] LustreError: 18250:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages
[ 5493.746402] Lustre: Unmounted lustre-client
[ 5494.215306] Lustre: Mounted lustre-client
[ 5495.429008] Lustre: DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000
[ 5509.250461] Lustre: lustre-MDT0000-mdc-ffff88012d0e7800: Connection to lustre-MDT0000 (at 192.168.202.134@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 5509.268184] Lustre: Skipped 7 previous similar messages
[ 5512.684600] Lustre: DEBUG MARKER: recovery-small test_155: @@@@@@ FAIL: total 2254