[ 3205.475913] Lustre: DEBUG MARKER: User quota (limit: 200)
[ 3207.172857] Lustre: DEBUG MARKER: Write 100M (buffered) ...
[ 3208.351470] LustreError: 8613:0:(osd_handler.c:698:osd_ro()) lustre-MDT0000: *** setting device osd-zfs read-only ***
[ 3208.738820] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000
[ 3209.363347] Lustre: DEBUG MARKER: Fail mds for 40 seconds
[ 3210.213879] Lustre: Failing over lustre-MDT0000
[ 3210.346724] Lustre: lustre-MDT0000: Not available for connect from 192.168.204.30@tcp (stopping)
[ 3210.366228] LustreError: 8987:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup
[ 3210.370473] LustreError: 8987:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message
[ 3210.455590] Lustre: server umount lustre-MDT0000 complete
[ 3223.377836] LustreError: MGC192.168.204.130@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 3223.576808] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 3223.583986] Lustre: Skipped 1 previous similar message
[ 3223.671746] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 3223.717229] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect
[ 3224.894525] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3227.482772] Lustre: 3294:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1746158038/real 1746158038]  req@ffff88009c0d3800 x1830976080139264/t0(0) o601->lustre-MDT0000-lwp-OST0000@0@lo:23/10 lens 336/336 e 0 to 1 dl 1746158054 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'lquota_wb_lustr.0' uid:0 gid:0
[ 3228.648445] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to  (at 0@lo)
[ 3229.514906] LustreError: 20193:0:(qsd_reint.c:617:qqi_reint_delayed()) lustre-OST0000: Delaying reintegration for qtype:0 until pending updates are flushed.
[ 3229.519541] LustreError: 20193:0:(qsd_reint.c:617:qqi_reint_delayed()) Skipped 6 previous similar messages
[ 3229.577486] LustreError: 9671:0:(tgt_handler.c:527:tgt_filter_recovery_request()) @@@ not permitted during recovery  req@ffff88009c0d3800 x1830976080151040/t0(0) o601->lustre-MDT0000-lwp-OST0000_UUID@0@lo:107/0 lens 336/0 e 0 to 0 dl 1746158067 ref 1 fl Interpret:/200/ffffffff rc 0/-1 job:'qsd_reint_0.lus.0' uid:0 gid:0
[ 3229.598029] LustreError: lustre-MDT0000-lwp-OST0000: operation quota_acquire to node 0@lo failed: rc = -11
[ 3230.598804] Lustre: 3296:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1746158041/real 1746158041]  req@ffff88009c0d3b80 x1830976080139520/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1746158057 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[ 3231.365301] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects
[ 3231.430791] Lustre: lustre-MDT0000: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted.
[ 3231.454222] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:680 to 0x240000400:705)
[ 3231.456710] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:676 to 0x280000400:705)
[ 3232.920519] Lustre: DEBUG MARKER: oleg430-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 3233.689117] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 3235.522579] Lustre: 3294:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1746158046/real 1746158046]  req@ffff88009c0d2300 x1830976080139904/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1746158062 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[ 3235.536670] Lustre: 3294:0:(client.c:2346:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[ 3236.577482] Lustre: DEBUG MARKER: (dd_pid=1616, time=0, timeout=600)
[ 3267.410908] Lustre: DEBUG MARKER: User quota (limit: 200)
[ 3269.851161] Lustre: DEBUG MARKER: Write 100M (directio) ...
[ 3271.724403] LustreError: 13081:0:(osd_handler.c:698:osd_ro()) lustre-MDT0000: *** setting device osd-zfs read-only ***
[ 3272.271300] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000
[ 3273.216278] Lustre: DEBUG MARKER: Fail mds for 40 seconds
[ 3274.475970] Lustre: Failing over lustre-MDT0000
[ 3274.656455] LustreError: 13848:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup
[ 3274.660867] LustreError: 13848:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages
[ 3274.783794] Lustre: server umount lustre-MDT0000 complete
[ 3288.209829] LustreError: MGC192.168.204.130@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 3288.486239] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 3288.522488] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect
[ 3289.777854] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3290.466677] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects
[ 3290.521971] Lustre: lustre-MDT0000: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted.
[ 3290.539069] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:707 to 0x240000400:737)
[ 3290.539074] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:676 to 0x280000400:737)
[ 3291.485765] Lustre: 3297:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1746158102/real 1746158102]  req@ffff8800a60ce680 x1830976080169984/t0(0) o601->lustre-MDT0000-lwp-OST0000@0@lo:23/10 lens 336/336 e 0 to 1 dl 1746158118 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'lquota_wb_lustr.0' uid:0 gid:0
[ 3291.496215] Lustre: 3297:0:(client.c:2346:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[ 3293.008083] Lustre: DEBUG MARKER: oleg430-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 3293.515688] LustreError: 10387:0:(qsd_reint.c:617:qqi_reint_delayed()) lustre-OST0000: Delaying reintegration for qtype:0 until pending updates are flushed.
[ 3293.584605] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 3295.550986] Lustre: 3294:0:(client.c:2346:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1746158106/real 1746158106]  req@ffff880132107100 x1830976080170240/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1746158122 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[ 3295.564876] Lustre: 3294:0:(client.c:2346:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[ 3300.296840] Lustre: DEBUG MARKER: (dd_pid=3989, time=4, timeout=600)
[ 3342.785662] Lustre: DEBUG MARKER: == sanity-quota test 19: Updating admin limits doesn't zero operational limits(b14790) ========================================================== 23:56:08 (1746158168)
[ 3355.795416] Lustre: DEBUG MARKER: Files for user (quota_usr), count=1:
[ 3356.614417] Lustre: DEBUG MARKER: Block quota isn't 0 (u:quota_usr:2).
[ 3358.854222] Lustre: DEBUG MARKER: Files for user (quota_usr), count=1:
[ 3359.517181] Lustre: DEBUG MARKER: Block quota isn't 0 (u:quota_usr:2).
[ 3382.682514] Lustre: DEBUG MARKER: == sanity-quota test 20: Test if setquota specifiers work properly (b15754) ========================================================== 23:56:48 (1746158208)
[ 3391.486841] Lustre: DEBUG MARKER: == sanity-quota test 21: Setquota while writing
[ 3400.266793] Lustre: DEBUG MARKER: Set limit(block:10G; file:1000000) for user: quota_usr
[ 3400.879604] Lustre: DEBUG MARKER: Set limit(block:10G; file:1000000) for group: quota_usr
[ 3401.565355] Lustre: DEBUG MARKER: Set limit(block:10G; file:) for project: 1000
[ 3402.335802] Lustre: DEBUG MARKER: Set quota for 1 times
[ 3404.090064] Lustre: DEBUG MARKER: Set quota for 2 times
[ 3405.916796] Lustre: DEBUG MARKER: Set quota for 3 times
[ 3407.945731] Lustre: DEBUG MARKER: Set quota for 4 times
[ 3410.182703] Lustre: DEBUG MARKER: Set quota for 5 times
[ 3412.283748] Lustre: DEBUG MARKER: Set quota for 6 times
[ 3414.625299] Lustre: DEBUG MARKER: Set quota for 7 times
[ 3416.789003] Lustre: DEBUG MARKER: Set quota for 8 times
[ 3419.179465] Lustre: DEBUG MARKER: Set quota for 9 times
[ 3421.516162] Lustre: DEBUG MARKER: Set quota for 10 times
[ 3423.696450] Lustre: DEBUG MARKER: Set quota for 11 times
[ 3425.881747] Lustre: DEBUG MARKER: Set quota for 12 times
[ 3428.217032] Lustre: DEBUG MARKER: Set quota for 13 times
[ 3430.225005] Lustre: DEBUG MARKER: Set quota for 14 times
[ 3432.248985] Lustre: DEBUG MARKER: Set quota for 15 times
[ 3463.857562] Lustre: DEBUG MARKER: == sanity-quota test 22: enable/disable quota by 'lctl conf_param/set_param -P' ========================================================== 23:58:09 (1746158289)
[ 3465.853306] LustreError: 27010:0:(qsd_reint.c:617:qqi_reint_delayed()) lustre-OST0001: Delaying reintegration for qtype:2 until pending updates are flushed.
[ 3465.860781] LustreError: 27010:0:(qsd_reint.c:617:qqi_reint_delayed()) Skipped 1 previous similar message
[ 3472.987768] LustreError: 27592:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup
[ 3472.992156] LustreError: 27592:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages
[ 3473.182128] Lustre: server umount lustre-MDT0000 complete
[ 3475.103572] LustreError: 7075:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1746158302 with bad export cookie 16262204251171272229
[ 3475.108263] LustreError: MGC192.168.204.130@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 3485.145473] Lustre: server umount lustre-OST0000 complete
[ 3496.925877] Lustre: server umount lustre-OST0001 complete
[ 3502.157146] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing load_modules_local
[ 3505.995499] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 3507.348363] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3508.633869] Lustre: Modifying parameter general.debug_raw_pointers in log params
[ 3511.015577] Lustre: 30568:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.130@tcp: no remote llog for lustre-sptlrpc, check MGS config
[ 3511.151455] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180
[ 3513.330446] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3516.136487] LustreError: 30572:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 3516.149316] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:739 to 0x240000400:769)
[ 3516.440838] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180
[ 3518.124696] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:740 to 0x280000400:769)
[ 3518.535755] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3522.351165] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 3524.003948] Lustre: Modifying parameter general.lod.*.mdt_hash in log params
[ 3541.208325] LustreError: 1341:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup
[ 3541.210514] LustreError: 1341:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages
[ 3541.298047] Lustre: server umount lustre-MDT0000 complete
[ 3543.294597] LustreError: 29558:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1746158370 with bad export cookie 16262204251171284556
[ 3543.300100] LustreError: MGC192.168.204.130@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 3553.336623] Lustre: server umount lustre-OST0000 complete
[ 3565.293726] Lustre: server umount lustre-OST0001 complete
[ 3572.666412] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing load_modules_local
[ 3576.353443] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 3578.178863] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3579.875412] Lustre: Modifying parameter general.debug_raw_pointers in log params
[ 3582.714437] Lustre: 4358:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.130@tcp: no remote llog for lustre-sptlrpc, check MGS config
[ 3582.720633] Lustre: 4358:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 1 previous similar message
[ 3585.337725] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3585.881757] LustreError: 4362:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 3585.912792] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:739 to 0x240000400:801)
[ 3589.107123] mount.lustre (5396) used greatest stack depth: 9488 bytes left
[ 3591.010351] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:740 to 0x280000400:801)
[ 3591.194942] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3594.955024] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 3596.513848] Lustre: Modifying parameter general.lod.*.mdt_hash in log params
[ 3606.232503] Lustre: DEBUG MARKER: == sanity-quota test 23: Quota should be honored with directIO (b16125) ========================================================== 00:00:31 (1746158431)
[ 3607.022560] Lustre: DEBUG MARKER: SKIP: sanity-quota test_23 Overwrite in place is not guaranteed to be space neutral on ZFS
[ 3607.953522] Lustre: DEBUG MARKER: == sanity-quota test 24: lfs draws an asterix when limit is reached (b16646) ========================================================== 00:00:33 (1746158433)
[ 3640.440144] Lustre: DEBUG MARKER: == sanity-quota test 25: check indexes versions ========== 00:01:06 (1746158466)
[ 3665.588249] Lustre: DEBUG MARKER: Write...
[ 3666.898352] Lustre: DEBUG MARKER: Write out of block quota ...
[ 3704.173458] Lustre: DEBUG MARKER: == sanity-quota test 27a: lfs quota/setquota should handle wrong arguments (b19612) ========================================================== 00:02:09 (1746158529)
[ 3707.531167] Lustre: DEBUG MARKER: == sanity-quota test 27b: lfs quota/setquota should handle user/group/project ID (b20200) ========================================================== 00:02:13 (1746158533)
[ 3713.400504] Lustre: DEBUG MARKER: == sanity-quota test 27c: lfs quota should support human-readable output ========================================================== 00:02:18 (1746158538)
[ 3718.324797] Lustre: DEBUG MARKER: == sanity-quota test 27d: lfs setquota should support fraction block limit ========================================================== 00:02:23 (1746158543)
[ 3722.462997] Lustre: DEBUG MARKER: == sanity-quota test 30: Hard limit updates should not reset grace times ========================================================== 00:02:28 (1746158548)
[ 3766.014030] Lustre: DEBUG MARKER: == sanity-quota test 33: Basic usage tracking for user
[ 3843.226527] Lustre: DEBUG MARKER: == sanity-quota test 34: Usage transfer for user
[ 3958.630005] Lustre: DEBUG MARKER: == sanity-quota test 35: Usage is still accessible across reboot ========================================================== 00:06:24 (1746158784)
[ 3993.386934] Lustre: DEBUG MARKER: Restart...
[ 3999.671296] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 3999.671727] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 3999.678668] Lustre: Skipped 3 previous similar messages
[ 4000.976440] LustreError: 29564:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup
[ 4000.980617] LustreError: 29564:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages
[ 4001.107193] Lustre: server umount lustre-MDT0000 complete
[ 4002.929043] LustreError: 3324:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1746158830 with bad export cookie 16262204251171286089
[ 4002.933597] LustreError: MGC192.168.204.130@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 4002.942427] LustreError: 3324:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) Skipped 2 previous similar messages
[ 4002.972774] Lustre: server umount lustre-OST0000 complete
[ 4004.857870] Lustre: server umount lustre-OST0001 complete
[ 4010.841373] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing load_modules_local
[ 4014.944159] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 4014.948004] Lustre: Skipped 2 previous similar messages
[ 4016.643622] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 4018.080820] Lustre: Modifying parameter general.debug_raw_pointers in log params
[ 4020.556414] Lustre: 32579:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.130@tcp: no remote llog for lustre-sptlrpc, check MGS config
[ 4020.559690] Lustre: 32579:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 1 previous similar message
[ 4023.097900] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 4024.701835] LustreError: 863:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 4024.720819] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:810 to 0x240000400:833)
[ 4028.322035] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:810 to 0x280000400:833)
[ 4029.348390] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 4033.759353] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 4035.680366] Lustre: Modifying parameter general.lod.*.mdt_hash in log params
[ 4071.462474] Lustre: DEBUG MARKER: == sanity-quota test 37: Quota accounted properly for file created by 'lfs setstripe' ========================================================== 00:08:16 (1746158896)
[ 4118.655782] Lustre: DEBUG MARKER: == sanity-quota test 38: Quota accounting iterator doesn't skip id entries ========================================================== 00:09:04 (1746158944)
[ 4845.603530] Lustre: DEBUG MARKER: == sanity-quota test 39: Project ID interface works correctly ========================================================== 00:21:11 (1746159671)
[ 4852.999184] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 4852.999811] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 4852.999812] Lustre: Skipped 1 previous similar message
[ 4853.011197] Lustre: Skipped 1 previous similar message
[ 4854.886562] LustreError: 21119:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup
[ 4854.889643] LustreError: 21119:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages
[ 4854.960761] Lustre: server umount lustre-MDT0000 complete
[ 4856.443972] LustreError: 31522:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1746159683 with bad export cookie 16262204251171292459
[ 4856.445556] LustreError: MGC192.168.204.130@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 4856.451974] LustreError: 31522:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) Skipped 2 previous similar messages
[ 4856.477666] Lustre: server umount lustre-OST0000 complete
[ 4857.934189] Lustre: server umount lustre-OST0001 complete
[ 4861.986147] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing load_modules_local
[ 4864.587146] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 4864.589869] Lustre: Skipped 2 previous similar messages
[ 4865.522285] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 4866.460635] Lustre: Modifying parameter general.debug_raw_pointers in log params
[ 4868.438990] Lustre: 24112:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.130@tcp: no remote llog for lustre-sptlrpc, check MGS config
[ 4868.444722] Lustre: 24112:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 1 previous similar message
[ 4868.606411] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180
[ 4870.524620] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 4873.023359] Lustre: 25150:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.130@tcp: no remote llog for lustre-sptlrpc, check MGS config
[ 4874.137113] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:5835 to 0x280000400:5857)
[ 4874.209971] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:5835 to 0x240000400:5857)
[ 4875.003774] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 4878.694697] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 4880.157517] Lustre: Modifying parameter general.lod.*.mdt_hash in log params
[ 4908.226052] Lustre: DEBUG MARKER: == sanity-quota test 40a: Hard link across different project ID ========================================================== 00:22:14 (1746159734)
[ 4938.172101] Lustre: DEBUG MARKER: == sanity-quota test 40b: Mv across different project ID ========================================================== 00:22:44 (1746159764)
[ 4967.591593] Lustre: DEBUG MARKER: == sanity-quota test 40c: Remote child Dir inherit project quota properly ========================================================== 00:23:13 (1746159793)
[ 4968.009588] Lustre: DEBUG MARKER: SKIP: sanity-quota test_40c needs >= 2 MDTs
[ 4968.487863] Lustre: DEBUG MARKER: == sanity-quota test 40d: Stripe Directory inherit project quota properly ========================================================== 00:23:14 (1746159794)
[ 4968.895128] Lustre: DEBUG MARKER: SKIP: sanity-quota test_40d needs >= 2 MDTs
[ 4969.400140] Lustre: DEBUG MARKER: == sanity-quota test 41: df should return projid-specific values ========================================================== 00:23:15 (1746159795)
[ 4978.184997] LustreError: 30553:0:(qsd_reint.c:617:qqi_reint_delayed()) lustre-MDT0000: Delaying reintegration for qtype:0 until pending updates are flushed.
[ 4978.190319] LustreError: 30553:0:(qsd_reint.c:617:qqi_reint_delayed()) Skipped 2 previous similar messages
[ 5006.201994] Lustre: DEBUG MARKER: == sanity-quota test 42: lfs quota should include default quota info ========================================================== 00:23:51 (1746159831)
[ 5032.572475] Lustre: DEBUG MARKER: == sanity-quota test 48: lfs quota --delete should delete quota project ID ========================================================== 00:24:18 (1746159858)
[ 5102.298244] Lustre: DEBUG MARKER: == sanity-quota test 49: lfs quota -a prints the quota usage for all quota IDs ========================================================== 00:25:27 (1746159927)
[ 5109.579303] Lustre: *** cfs_fail_loc=a09, val=0***
[ 5109.584154] Lustre: Skipped 1 previous similar message
[ 5110.591283] Lustre: *** cfs_fail_loc=a09, val=0***
[ 5110.592211] Lustre: Skipped 105 previous similar messages
[ 5112.595187] Lustre: *** cfs_fail_loc=a09, val=0***
[ 5112.598063] Lustre: Skipped 210 previous similar messages
[ 5116.601384] Lustre: *** cfs_fail_loc=a09, val=0***
[ 5116.603174] Lustre: Skipped 380 previous similar messages
[ 5124.629150] Lustre: *** cfs_fail_loc=a09, val=0***
[ 5124.631027] Lustre: Skipped 779 previous similar messages
[ 5140.630137] Lustre: *** cfs_fail_loc=a09, val=0***
[ 5140.636925] Lustre: Skipped 1502 previous similar messages
[ 5172.635520] Lustre: *** cfs_fail_loc=a09, val=0***
[ 5172.641956] Lustre: Skipped 2956 previous similar messages
[ 5480.107713] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 5480.109706] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 5480.109707] Lustre: Skipped 1 previous similar message
[ 5480.135995] Lustre: Skipped 1 previous similar message
[ 5485.113111] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 5485.115575] Lustre: Skipped 2 previous similar messages
[ 5485.949183] LustreError: 13862:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup
[ 5485.958178] LustreError: 13862:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages
[ 5486.155992] Lustre: server umount lustre-MDT0000 complete
[ 5488.894313] LustreError: 30389:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1746160316 with bad export cookie 16262204251173048241
[ 5488.902387] LustreError: MGC192.168.204.130@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 5489.091892] Lustre: server umount lustre-OST0000 complete
[ 5491.686970] Lustre: server umount lustre-OST0001 complete
[ 5494.930565] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_hostid
[ 5497.891424] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing load_modules_local
[ 5502.210033]  vdc: vdc1 vdc9
[ 5506.710412]  vde: vde1 vde9
[ 5510.694555]  vdf: vdf1 vdf9
[ 5516.179396] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing load_modules_local
[ 5519.513625] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000
[ 5519.612083] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space.
[ 5519.652261] Lustre: lustre-MDT0000: new disk, initializing
[ 5519.844972] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 5519.849663] Lustre: Skipped 1 previous similar message
[ 5519.883518] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt
[ 5521.314308] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 5524.166346] Lustre: Modifying parameter general.debug_raw_pointers in log params
[ 5526.966088] Lustre: 20071:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.130@tcp: no remote llog for lustre-sptlrpc, check MGS config
[ 5526.985218] Lustre: lustre-OST0000: new disk, initializing
[ 5526.989312] Lustre: srv-lustre-OST0000: No data found on store. Initialize space.
[ 5526.995973] Lustre: Skipped 1 previous similar message
[ 5527.055610] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180
[ 5528.124451] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost
[ 5528.129146] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost]
[ 5528.213173] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400
[ 5529.466765] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 5534.339057] Lustre: 21387:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.204.130@tcp: no remote llog for lustre-sptlrpc, check MGS config
[ 5534.349914] Lustre: lustre-OST0001: new disk, initializing
[ 5534.352412] Lustre: srv-lustre-OST0001: No data found on store. Initialize space.
[ 5534.395726] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180
[ 5536.207531] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost
[ 5536.214664] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost]
[ 5536.297723] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400
[ 5536.723414] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 5541.714630] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 5543.346297] Lustre: Setting parameter general.lod.*.mdt_hash in log params
[ 5567.039671] Lustre: DEBUG MARKER: == sanity-quota test 50: Test if lfs find --projid works ========================================================== 00:33:12 (1746160392)
[ 5600.095343] Lustre: DEBUG MARKER: == sanity-quota test 51: Test project accounting with mv/cp ========================================================== 00:33:45 (1746160425)
[ 5629.501354] Lustre: DEBUG MARKER: == sanity-quota test 52: Rename normal file across project ID ========================================================== 00:34:15 (1746160455)
[ 5642.481708] Lustre: DEBUG MARKER: rename directory return 255
[ 5664.902923] Lustre: DEBUG MARKER: == sanity-quota test 53: Project inherit attribute could be cleared ========================================================== 00:34:50 (1746160490)
[ 5681.734066] Lustre: DEBUG MARKER: == sanity-quota test 54: basic lfs project interface test ========================================================== 00:35:07 (1746160507)
[ 5707.030769] Lustre: DEBUG MARKER: == sanity-quota test 55: Chgrp should be affected by group quota ========================================================== 00:35:32 (1746160532)
[ 5737.800952] LustreError: 18865:0:(qmt_entry.c:552:qmt_adjust_edquot()) $$$ set revoke_time explicitly  qmt:lustre-QMT0000 pool:dt-0x0 id:60001 enforced:1 hard:51200 soft:0 granted:51200 time:0 qunit: 1024 edquot:0 may_rel:0 revoke:0 default:no
[ 5759.615728] Lustre: DEBUG MARKER: == sanity-quota test 56: lfs quota -t should work well === 00:36:25 (1746160585)
[ 5777.968083] Lustre: DEBUG MARKER: == sanity-quota test 57: lfs project could tolerate errors ========================================================== 00:36:43 (1746160603)
[ 5805.146741] Lustre: DEBUG MARKER: == sanity-quota test 58: project ID should be kept for new mirrors created by FID ========================================================== 00:37:10 (1746160630)
[ 5884.484662] Lustre: DEBUG MARKER: == sanity-quota test 59: lfs project dosen't crash kernel with project disabled ========================================================== 00:38:30 (1746160710)
[ 5885.043824] Lustre: DEBUG MARKER: SKIP: sanity-quota test_59 ldiskfs only test
[ 5885.704566] Lustre: DEBUG MARKER: == sanity-quota test 60: Test quota for root with setgid ========================================================== 00:38:31 (1746160711)
[ 5894.540253] LustreError: 16269:0:(qsd_reint.c:617:qqi_reint_delayed()) lustre-MDT0000: Delaying reintegration for qtype:0 until pending updates are flushed.
[ 5894.548412] LustreError: 16269:0:(qsd_reint.c:617:qqi_reint_delayed()) Skipped 4 previous similar messages
[ 5922.947107] Lustre: DEBUG MARKER: SKIP: sanity-quota test_61 skipping SLOW test 61
[ 5923.571698] Lustre: DEBUG MARKER: == sanity-quota test 62: Project inherit should be only changed by root ========================================================== 00:39:09 (1746160749)
[ 5941.852768] Lustre: DEBUG MARKER: SKIP: sanity-quota test_63 skipping excluded test 63
[ 5942.411130] Lustre: DEBUG MARKER: == sanity-quota test 64: lfs project on non dir/files should succeed ========================================================== 00:39:28 (1746160768)
[ 5970.860425] Lustre: DEBUG MARKER: SKIP: sanity-quota test_65 skipping excluded test 65
[ 5971.615557] Lustre: DEBUG MARKER: == sanity-quota test 66: nonroot user can not change project state in default ========================================================== 00:39:57 (1746160797)
[ 6003.868140] Lustre: DEBUG MARKER: == sanity-quota test 67: quota pools recalculation ======= 00:40:29 (1746160829)
[ 6004.626236] Lustre: DEBUG MARKER: SKIP: sanity-quota test_67 ZFS grants some block space together with inode
[ 6005.499576] Lustre: DEBUG MARKER: == sanity-quota test 68: slave number in quota pool changed after each add/remove OST ========================================================== 00:40:31 (1746160831)
[ 6023.251842] LustreError: 23230:0:(mgs_handler.c:848:mgs_iocontrol_pool()) OBD_IOC_POOL err -17, cmd CE021 for pool lustre.qpool1
[ 6050.102974] Lustre: DEBUG MARKER: == sanity-quota test 69: EDQUOT at one of pools shouldn't affect DOM ========================================================== 00:41:15 (1746160875)
[ 6061.150771] LustreError: 25660:0:(qsd_reint.c:617:qqi_reint_delayed()) lustre-MDT0000: Delaying reintegration for qtype:2 until pending updates are flushed.
[ 6061.154884] LustreError: 25660:0:(qsd_reint.c:617:qqi_reint_delayed()) Skipped 5 previous similar messages
[ 6068.212474] Lustre: DEBUG MARKER: User quota (block hardlimit:200 MB)
[ 6068.962907] Lustre: DEBUG MARKER: User quota (block hardlimit:10 MB)
[ 6153.518612] Lustre: DEBUG MARKER: == sanity-quota test 70a: check lfs setquota/quota with a pool option ========================================================== 00:42:59 (1746160979)
[ 6183.896716] Lustre: DEBUG MARKER: == sanity-quota test 70b: lfs setquota pool works properly ========================================================== 00:43:29 (1746161009)
[ 6201.589871] Lustre: DEBUG MARKER: == sanity-quota test 71a: Check PFL with quota pools ===== 00:43:47 (1746161027)
[ 6202.148009] Lustre: DEBUG MARKER: SKIP: sanity-quota test_71a ZFS grants some block space together with inode
[ 6202.932132] Lustre: DEBUG MARKER: == sanity-quota test 71b: Check SEL with quota pools ===== 00:43:48 (1746161028)
[ 6203.668586] Lustre: DEBUG MARKER: SKIP: sanity-quota test_71b ZFS grants some block space together with inode
[ 6204.542420] Lustre: DEBUG MARKER: == sanity-quota test 72: lfs quota --pool prints only pool's OSTs ========================================================== 00:43:50 (1746161030)
[ 6213.811783] Lustre: DEBUG MARKER: User quota (block hardlimit:50 MB)
[ 6221.314208] Lustre: DEBUG MARKER: Write...
[ 6222.274522] Lustre: DEBUG MARKER: Write out of block quota ...
[ 6254.426905] Lustre: DEBUG MARKER: == sanity-quota test 73a: default limits at OST Pool Quotas ========================================================== 00:44:40 (1746161080)
[ 6271.194921] Lustre: DEBUG MARKER: set to use default quota
[ 6271.986614] Lustre: DEBUG MARKER: set default quota
[ 6272.833463] Lustre: DEBUG MARKER: get default quota
[ 6275.066417] Lustre: DEBUG MARKER: Test not out of quota
[ 6277.233851] Lustre: DEBUG MARKER: Test out of quota
[ 6282.599465] Lustre: DEBUG MARKER: Increase default quota
[ 6293.303218] Lustre: DEBUG MARKER: Set quota to override default quota
[ 6293.325033] LustreError: 21831:0:(qmt_entry.c:552:qmt_adjust_edquot()) $$$ set revoke_time explicitly  qmt:lustre-QMT0000 pool:dt-qpool1 id:60000 enforced:1 hard:20480 soft:20480 granted:45063 time:1746765920 qunit: 1024 edquot:0 may_rel:0 revoke:0 default:no
[ 6297.698383] Lustre: DEBUG MARKER: Set to use default quota again
[ 6308.938770] Lustre: DEBUG MARKER: Cleanup
[ 6353.295062] Lustre: DEBUG MARKER: == sanity-quota test 73b: default OST Pool Quotas limit for new user ========================================================== 00:46:19 (1746161179)
[ 6368.147278] Lustre: DEBUG MARKER: set default quota for qpool1
[ 6368.757527] Lustre: DEBUG MARKER: Write from user that hasn't lqe
[ 6401.286228] Lustre: DEBUG MARKER: == sanity-quota test 74: check quota pools per user ====== 00:47:06 (1746161226)
[ 6449.964289] Lustre: DEBUG MARKER: == sanity-quota test 75: nodemap squashed root respects quota enforcement ========================================================== 00:47:55 (1746161275)
[ 6537.035623] Lustre: DEBUG MARKER: Write...
[ 6538.926345] Lustre: DEBUG MARKER: Write out of block quota ...
[ 6649.136857] Lustre: DEBUG MARKER: == sanity-quota test 76: project ID 4294967295 should be not allowed ========================================================== 00:51:14 (1746161474)
[ 6677.876196] Lustre: DEBUG MARKER: == sanity-quota test 77: lfs setquota should fail in Lustre mount with 'ro' ========================================================== 00:51:43 (1746161503)
[ 6682.241001] Lustre: DEBUG MARKER: == sanity-quota test 78A: Check fallocate increase quota usage ========================================================== 00:51:47 (1746161507)
[ 6683.347357] Lustre: DEBUG MARKER: SKIP: sanity-quota test_78A need >= 2.13.57 and ldiskfs for fallocate
[ 6684.465954] Lustre: DEBUG MARKER: == sanity-quota test 78a: Check fallocate increase projectid usage ========================================================== 00:51:49 (1746161509)
[ 6685.337398] Lustre: DEBUG MARKER: SKIP: sanity-quota test_78a need >= 2.13.57 and ldiskfs for fallocate
[ 6686.504275] Lustre: DEBUG MARKER: == sanity-quota test 79: access to non-existed dt-pool/info doesn't cause a panic ========================================================== 00:51:51 (1746161511)
[ 6700.446021] Lustre: DEBUG MARKER: == sanity-quota test 80: check for EDQUOT after OST failover ========================================================== 00:52:05 (1746161525)
[ 6701.398831] Lustre: DEBUG MARKER: SKIP: sanity-quota test_80 ZFS grants some block space together with inode
[ 6702.817121] Lustre: DEBUG MARKER: == sanity-quota test 81: Race qmt_start_pool_recalc with qmt_pool_free ========================================================== 00:52:08 (1746161528)
[ 6713.879614] Lustre: DEBUG MARKER: User quota (block hardlimit:20 MB)
[ 6719.374378] LustreError: 28554:0:(qmt_pool.c:1295:qmt_pool_recalc()) cfs_fail_timeout id a07 sleeping for 10000ms
[ 6723.205937] Lustre: lustre-MDT0000: Not available for connect from 192.168.204.30@tcp (stopping)
[ 6724.055662] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 6724.066527] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 6724.071367] Lustre: Skipped 1 previous similar message
[ 6728.202914] Lustre: lustre-MDT0000: Not available for connect from 192.168.204.30@tcp (stopping)
[ 6729.436753] LustreError: 28554:0:(qmt_pool.c:1295:qmt_pool_recalc()) cfs_fail_timeout id a07 awake
[ 6729.545878] LustreError: 28694:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup
[ 6729.548570] LustreError: 28694:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages
[ 6729.690289] Lustre: server umount lustre-MDT0000 complete
[ 6733.512370] LustreError: MGC192.168.204.130@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 6733.777500] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 6733.840453] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:27 to 0x280000400:65)
[ 6733.846926] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:27 to 0x240000400:65)
[ 6735.349981] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 6738.763559] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to  (at 0@lo)
[ 6738.766331] Lustre: Skipped 3 previous similar messages
[ 6771.086388] Lustre: DEBUG MARKER: == sanity-quota test 82: verify more than 8 qids for single operation ========================================================== 00:53:16 (1746161596)
[ 6789.854204] Lustre: DEBUG MARKER: == sanity-quota test 83: Setting default quota shouldn't affect grace time ========================================================== 00:53:35 (1746161615)
[ 6809.473751] Lustre: DEBUG MARKER: == sanity-quota test 84: Reset quota should fix the insane granted quota ========================================================== 00:53:55 (1746161635)
[ 6826.326231] LustreError: 20241:0:(qsd_reint.c:617:qqi_reint_delayed()) lustre-OST0000: Delaying reintegration for qtype:0 until pending updates are flushed.
[ 6826.332830] LustreError: 20241:0:(qsd_reint.c:617:qqi_reint_delayed()) Skipped 2 previous similar messages
[ 6839.275035] Lustre: *** cfs_fail_loc=a08, val=0***
[ 6839.279044] Lustre: Skipped 2061 previous similar messages
[ 6839.284956] Lustre: *** cfs_fail_loc=a08, val=0***
[ 6909.012839] Lustre: DEBUG MARKER: == sanity-quota test 85: do not hung at write with the least_qunit ========================================================== 00:55:34 (1746161734)
[ 6963.971131] Lustre: DEBUG MARKER: == sanity-quota test 86: Pre-acquired quota should be released if quota is over limit ========================================================== 00:56:29 (1746161789)
[ 7085.566457] LustreError: 29404:0:(qmt_entry.c:552:qmt_adjust_edquot()) $$$ set revoke_time explicitly  qmt:lustre-QMT0000 pool:md-0x0 id:60000 enforced:1 hard:2048 soft:2048 granted:65536 time:1746766712 qunit: 1024 edquot:0 may_rel:0 revoke:0 default:no
[ 7251.280922] LustreError: 17783:0:(qmt_entry.c:552:qmt_adjust_edquot()) $$$ set revoke_time explicitly  qmt:lustre-QMT0000 pool:md-0x0 id:60000 enforced:1 hard:2048 soft:2048 granted:65536 time:1746766878 qunit: 1024 edquot:0 may_rel:0 revoke:0 default:no
[ 7421.860584] LustreError: 13246:0:(qmt_entry.c:552:qmt_adjust_edquot()) $$$ set revoke_time explicitly  qmt:lustre-QMT0000 pool:md-0x0 id:1000 enforced:1 hard:2048 soft:2048 granted:65536 time:1746767049 qunit: 1024 edquot:0 may_rel:0 revoke:0 default:no
[ 7493.760567] Lustre: DEBUG MARKER: == sanity-quota test 87: lfs quota -a should print default quota setting ========================================================== 01:05:19 (1746162319)
[ 7501.581455] Lustre: *** cfs_fail_loc=a09, val=0***
[ 7502.594278] Lustre: *** cfs_fail_loc=a09, val=0***
[ 7502.595938] Lustre: Skipped 87 previous similar messages
[ 7504.593744] Lustre: *** cfs_fail_loc=a09, val=0***
[ 7504.595931] Lustre: Skipped 189 previous similar messages
[ 7508.610334] Lustre: *** cfs_fail_loc=a09, val=0***
[ 7508.611858] Lustre: Skipped 565 previous similar messages
[ 7544.554318] Lustre: lustre-MDT0000: Not available for connect from 192.168.204.30@tcp (stopping)
[ 7544.558012] Lustre: Skipped 2 previous similar messages
[ 7545.115206] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 7545.116944] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 7545.130627] Lustre: Skipped 2 previous similar messages
[ 7547.127224] LustreError: 31288:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup
[ 7547.131075] LustreError: 31288:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages
[ 7547.235735] Lustre: server umount lustre-MDT0000 complete
[ 7550.745811] LustreError: MGC192.168.204.130@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 7551.009501] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 7551.060233] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:67 to 0x280000400:97)
[ 7551.060363] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:67 to 0x240000400:97)
[ 7552.301426] Lustre: DEBUG MARKER: oleg430-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 7564.979251] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to  (at 0@lo)
[ 7564.987587] Lustre: Skipped 1 previous similar message
[ 7575.027476] Lustre: DEBUG MARKER: == sanity-quota test 88: Writing over quota should not hang ========================================================== 01:06:40 (1746162400)
[ 7575.774202] Lustre: DEBUG MARKER: SKIP: sanity-quota test_88 require client with >4k pages
[ 7576.732157] Lustre: DEBUG MARKER: == sanity-quota test 89: Show default quota with squash_uid ========================================================== 01:06:42 (1746162402)
[ 7617.468843] Lustre: DEBUG MARKER: == sanity-quota test 90a: lfs quota should work without mount point ========================================================== 01:07:23 (1746162443)
[ 7647.445348] Lustre: DEBUG MARKER: == sanity-quota test 90b: lfs quota should work with multiple mount points ========================================================== 01:07:53 (1746162473)
[ 7656.719814] Lustre: DEBUG MARKER: sanity-quota test_90b: @@@@@@ FAIL: quota info from /mnt/lustre not ' /mnt/lustre root 11715 0 0 - 375 0 0 -