[ 3086.463321] Lustre: DEBUG MARKER: User quota (limit: 200)
[ 3088.426114] Lustre: DEBUG MARKER: Write 100M (buffered) ...
[ 3089.516559] LustreError: 102234:0:(osd_handler.c:721:osd_ro()) lustre-MDT0000: *** setting device osd-zfs read-only ***
[ 3089.835252] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000
[ 3090.381119] Lustre: DEBUG MARKER: Fail mds for 40 seconds
[ 3091.131850] Lustre: Failing over lustre-MDT0000
[ 3091.547689] LustreError: 102427:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup
[ 3091.549420] LustreError: 102427:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message
[ 3091.627648] Lustre: server umount lustre-MDT0000 complete
[ 3105.174310] LustreError: MGC192.168.204.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 3105.248214] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 3105.248623] LustreError: 102828:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 3105.255439] Lustre: Skipped 1 previous similar message
[ 3105.489677] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 3105.519231] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect
[ 3105.600871] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects
[ 3105.660878] Lustre: lustre-MDT0000: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted.
[ 3105.680285] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:1257 to 0x240000400:1281)
[ 3105.680311] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:1254 to 0x280000400:1281)
[ 3107.335345] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3107.487171] Lustre: 3290:0:(client.c:2453:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1755639955/real 1755639955]  req@ffff9beffcaf7b80 x1840918741939840/t0(0) o601->lustre-MDT0000-lwp-OST0000@0@lo:23/10 lens 336/336 e 0 to 1 dl 1755639971 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'lquota_wb_lustr.0' uid:0 gid:0 projid:4294967295
[ 3110.559411] Lustre: 3292:0:(client.c:2453:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1755639958/real 1755639958]  req@ffff9befe78c4380 x1840918741940096/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1755639974 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
[ 3110.885130] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to  (at 0@lo)
[ 3111.339664] Lustre: DEBUG MARKER: oleg450-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 3112.125808] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 3112.638070] LustreError: 85814:0:(qsd_reint.c:618:qqi_reint_delayed()) lustre-OST0000: Delaying reintegration for qtype:0 until pending updates are flushed.
[ 3112.644401] LustreError: 85814:0:(qsd_reint.c:618:qqi_reint_delayed()) Skipped 6 previous similar messages
[ 3115.743197] Lustre: 3293:0:(client.c:2453:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1755639963/real 1755639963]  req@ffff9beec3c53480 x1840918741940608/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1755639979 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
[ 3115.756696] Lustre: 3293:0:(client.c:2453:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[ 3120.342669] Lustre: DEBUG MARKER: (dd_pid=97240, time=5, timeout=600)
[ 3150.201030] Lustre: DEBUG MARKER: User quota (limit: 200)
[ 3152.213314] Lustre: DEBUG MARKER: Write 100M (directio) ...
[ 3153.756700] LustreError: 104709:0:(osd_handler.c:721:osd_ro()) lustre-MDT0000: *** setting device osd-zfs read-only ***
[ 3154.130091] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000
[ 3154.888121] Lustre: DEBUG MARKER: Fail mds for 40 seconds
[ 3155.772142] Lustre: Failing over lustre-MDT0000
[ 3156.021624] LustreError: lustre-MDT0000-lwp-OST0000: operation quota_acquire to node 0@lo failed: rc = -107
[ 3156.024086] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 3156.096905] LustreError: 104902:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup
[ 3156.100173] LustreError: 104902:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages
[ 3156.187195] Lustre: server umount lustre-MDT0000 complete
[ 3169.629939] LustreError: MGC192.168.204.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 3169.924295] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 3169.949149] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect
[ 3171.709933] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3172.161668] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects
[ 3172.236558] Lustre: lustre-MDT0000: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted.
[ 3172.262186] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:1254 to 0x280000400:1313)
[ 3172.263490] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:1283 to 0x240000400:1313)
[ 3173.023312] Lustre: 3293:0:(client.c:2453:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1755640020/real 1755640020]  req@ffff9beeca6bad80 x1840918741971200/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1755640036 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
[ 3173.037513] Lustre: 3293:0:(client.c:2453:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[ 3175.761280] Lustre: DEBUG MARKER: oleg450-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 3176.627675] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 3177.150141] LustreError: 85814:0:(qsd_reint.c:618:qqi_reint_delayed()) lustre-OST0000: Delaying reintegration for qtype:0 until pending updates are flushed.
[ 3177.439298] Lustre: 3293:0:(client.c:2453:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1755640025/real 1755640025]  req@ffff9beffca86a00 x1840918741971968/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1755640041 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
[ 3188.074396] Lustre: DEBUG MARKER: (dd_pid=99471, time=8, timeout=600)
[ 3229.639953] Lustre: DEBUG MARKER: == sanity-quota test 19: Updating admin limits doesn't zero operational limits(b14790) ========================================================== 17:48:12 (1755640092)
[ 3242.272784] Lustre: DEBUG MARKER: Files for user (quota_usr), count=1:
[ 3243.040454] Lustre: DEBUG MARKER: Block quota isn't 0 (u:quota_usr:2).
[ 3245.367176] Lustre: DEBUG MARKER: Files for user (quota_usr), count=1:
[ 3246.216868] Lustre: DEBUG MARKER: Block quota isn't 0 (u:quota_usr:2).
[ 3268.120169] Lustre: DEBUG MARKER: == sanity-quota test 20: Test if setquota specifiers work properly (b15754) ========================================================== 17:48:50 (1755640130)
[ 3278.213621] Lustre: DEBUG MARKER: == sanity-quota test 21: Setquota while writing
[ 3288.040317] Lustre: DEBUG MARKER: Set limit(block:10G; file:1000000) for user: quota_usr
[ 3288.858534] Lustre: DEBUG MARKER: Set limit(block:10G; file:1000000) for group: quota_usr
[ 3289.695386] Lustre: DEBUG MARKER: Set limit(block:10G; file:) for project: 1000
[ 3290.288643] Lustre: DEBUG MARKER: Set quota for 1 times
[ 3292.015356] Lustre: DEBUG MARKER: Set quota for 2 times
[ 3293.740676] Lustre: DEBUG MARKER: Set quota for 3 times
[ 3295.425338] Lustre: DEBUG MARKER: Set quota for 4 times
[ 3297.094243] Lustre: DEBUG MARKER: Set quota for 5 times
[ 3298.779476] Lustre: DEBUG MARKER: Set quota for 6 times
[ 3300.493517] Lustre: DEBUG MARKER: Set quota for 7 times
[ 3302.184747] Lustre: DEBUG MARKER: Set quota for 8 times
[ 3303.910606] Lustre: DEBUG MARKER: Set quota for 9 times
[ 3305.652718] Lustre: DEBUG MARKER: Set quota for 10 times
[ 3307.203818] Lustre: DEBUG MARKER: Set quota for 11 times
[ 3308.910621] Lustre: DEBUG MARKER: Set quota for 12 times
[ 3310.642515] Lustre: DEBUG MARKER: Set quota for 13 times
[ 3312.355101] Lustre: DEBUG MARKER: Set quota for 14 times
[ 3314.043844] Lustre: DEBUG MARKER: Set quota for 15 times
[ 3315.774427] Lustre: DEBUG MARKER: Set quota for 16 times
[ 3317.422094] Lustre: DEBUG MARKER: Set quota for 17 times
[ 3319.161777] Lustre: DEBUG MARKER: Set quota for 18 times
[ 3349.563612] Lustre: DEBUG MARKER: == sanity-quota test 22: enable/disable quota by 'lctl conf_param/set_param -P' ========================================================== 17:50:12 (1755640212)
[ 3351.270219] LustreError: 111440:0:(qsd_reint.c:618:qqi_reint_delayed()) lustre-OST0001: Delaying reintegration for qtype:2 until pending updates are flushed.
[ 3351.279866] LustreError: 111440:0:(qsd_reint.c:618:qqi_reint_delayed()) Skipped 1 previous similar message
[ 3358.319834] LustreError: 111831:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup
[ 3358.322089] LustreError: 111831:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages
[ 3358.405757] Lustre: server umount lustre-MDT0000 complete
[ 3360.448082] LustreError: 5732:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755640224 with bad export cookie 2601557930964739219
[ 3360.454728] LustreError: MGC192.168.204.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 3371.015143] Lustre: server umount lustre-OST0000 complete
[ 3383.325071] Lustre: server umount lustre-OST0001 complete
[ 3389.955377] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing load_modules_local
[ 3394.312919] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 3396.022508] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3397.449287] Lustre: 113696:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17
[ 3400.110893] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180
[ 3402.815188] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3404.193382] LustreError: 114019:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 3404.200330] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:1316 to 0x240000400:1345)
[ 3404.206538] LustreError: 114019:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message
[ 3406.447473] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180
[ 3408.191252] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:1315 to 0x280000400:1345)
[ 3409.178311] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3413.312813] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 3415.015059] Lustre: 115466:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17
[ 3431.904091] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 3431.907459] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 3431.908630] Lustre: Skipped 3 previous similar messages
[ 3431.910615] LustreError: 116271:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup
[ 3431.910626] LustreError: 116271:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages
[ 3431.916583] LustreError: 113284:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 3432.001517] Lustre: server umount lustre-MDT0000 complete
[ 3433.991411] LustreError: 113270:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755640297 with bad export cookie 2601557930964746835
[ 3433.993605] LustreError: MGC192.168.204.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 3433.995052] LustreError: 113270:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages
[ 3434.041753] Lustre: server umount lustre-OST0000 complete
[ 3435.914827] Lustre: server umount lustre-OST0001 complete
[ 3442.366226] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing load_modules_local
[ 3446.606991] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 3448.390006] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3449.699648] Lustre: 118116:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17
[ 3454.935268] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3457.380951] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:1316 to 0x240000400:1377)
[ 3459.574406] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:1315 to 0x280000400:1377)
[ 3460.393356] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3464.439714] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 3466.216402] Lustre: 119890:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17
[ 3476.031572] Lustre: DEBUG MARKER: == sanity-quota test 23: Quota should be honored with directIO (b16125) ========================================================== 17:52:18 (1755640338)
[ 3476.720306] Lustre: DEBUG MARKER: SKIP: sanity-quota test_23 Overwrite in place is not guaranteed to be space neutral on ZFS
[ 3477.458434] Lustre: DEBUG MARKER: == sanity-quota test 24: lfs draws an asterix when limit is reached (b16646) ========================================================== 17:52:20 (1755640340)
[ 3513.990206] Lustre: DEBUG MARKER: == sanity-quota test 25: check indexes versions ========== 17:52:56 (1755640376)
[ 3537.041097] Lustre: DEBUG MARKER: Write...
[ 3538.224406] Lustre: DEBUG MARKER: Write out of block quota ...
[ 3577.150533] Lustre: DEBUG MARKER: == sanity-quota test 27a: lfs quota/setquota should handle wrong arguments (b19612) ========================================================== 17:53:59 (1755640439)
[ 3579.981383] Lustre: DEBUG MARKER: == sanity-quota test 27b: lfs quota/setquota should handle user/group/project ID (b20200) ========================================================== 17:54:02 (1755640442)
[ 3585.731242] Lustre: DEBUG MARKER: == sanity-quota test 27c: lfs quota should support human-readable output ========================================================== 17:54:08 (1755640448)
[ 3589.738973] Lustre: DEBUG MARKER: == sanity-quota test 27d: lfs setquota should support fraction block limit ========================================================== 17:54:12 (1755640452)
[ 3594.042521] Lustre: DEBUG MARKER: == sanity-quota test 30: Hard limit updates should not reset grace times ========================================================== 17:54:16 (1755640456)
[ 3640.828382] Lustre: DEBUG MARKER: == sanity-quota test 33: Basic usage tracking for user
[ 3726.009865] Lustre: DEBUG MARKER: == sanity-quota test 34: Usage transfer for user
[ 3840.810475] Lustre: DEBUG MARKER: == sanity-quota test 35: Usage is still accessible across reboot ========================================================== 17:58:23 (1755640703)
[ 3878.513983] Lustre: DEBUG MARKER: Restart...
[ 3882.977581] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 3882.981840] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 3882.995766] Lustre: Skipped 1 previous similar message
[ 3886.208783] LustreError: 133487:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup
[ 3886.212915] LustreError: 133487:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages
[ 3886.314543] Lustre: server umount lustre-MDT0000 complete
[ 3888.342385] LustreError: 122775:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755640751 with bad export cookie 2601557930964748368
[ 3888.347860] LustreError: MGC192.168.204.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 3888.352379] LustreError: 122775:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages
[ 3888.405283] Lustre: server umount lustre-OST0000 complete
[ 3890.504632] Lustre: server umount lustre-OST0001 complete
[ 3897.627392] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing load_modules_local
[ 3902.067372] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 3902.071106] Lustre: Skipped 2 previous similar messages
[ 3903.987736] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3905.458482] Lustre: 135338:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17
[ 3910.849541] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3912.289480] LustreError: 135662:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 3912.301198] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:1387 to 0x240000400:1409)
[ 3912.301461] LustreError: 135662:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message
[ 3916.347808] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:1385 to 0x280000400:1409)
[ 3917.203325] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3921.395571] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 3923.208375] Lustre: 137110:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17
[ 3960.322194] Lustre: DEBUG MARKER: == sanity-quota test 37: Quota accounted properly for file created by 'lfs setstripe' ========================================================== 18:00:23 (1755640823)
[ 4009.748469] Lustre: DEBUG MARKER: == sanity-quota test 38: Quota accounting iterator doesn't skip id entries ========================================================== 18:01:12 (1755640872)
[ 4681.693649] Lustre: DEBUG MARKER: == sanity-quota test 39: Project ID interface works correctly ========================================================== 18:12:24 (1755641544)
[ 4692.421571] LustreError: 142080:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup
[ 4692.424423] LustreError: 142080:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages
[ 4692.515599] Lustre: server umount lustre-MDT0000 complete
[ 4694.542879] LustreError: 135683:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755641558 with bad export cookie 2601557930964754759
[ 4694.549852] LustreError: MGC192.168.204.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 4704.789720] Lustre: server umount lustre-OST0000 complete
[ 4716.854921] Lustre: server umount lustre-OST0001 complete
[ 4722.998178] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing load_modules_local
[ 4727.003818] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 4727.007769] Lustre: Skipped 2 previous similar messages
[ 4728.654403] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 4729.899172] Lustre: 143930:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17
[ 4732.543981] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180
[ 4735.089753] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 4736.608695] LustreError: 144762:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 4736.615505] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:6411 to 0x240000400:6433)
[ 4740.092786] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:6411 to 0x280000400:6433)
[ 4741.063004] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 4745.225344] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 4746.918912] Lustre: 145707:0:(mgs_llog.c:1345:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17
[ 4778.861348] Lustre: DEBUG MARKER: == sanity-quota test 40a: Hard link across different project ID ========================================================== 18:14:01 (1755641641)
[ 4806.411901] Lustre: DEBUG MARKER: == sanity-quota test 40b: Mv across different project ID ========================================================== 18:14:29 (1755641669)
[ 4834.534256] Lustre: DEBUG MARKER: == sanity-quota test 40c: Remote child Dir inherit project quota properly ========================================================== 18:14:57 (1755641697)
[ 4835.266565] Lustre: DEBUG MARKER: SKIP: sanity-quota test_40c needs >= 2 MDTs
[ 4836.110097] Lustre: DEBUG MARKER: == sanity-quota test 40d: Stripe Directory inherit project quota properly ========================================================== 18:14:58 (1755641698)
[ 4836.864985] Lustre: DEBUG MARKER: SKIP: sanity-quota test_40d needs >= 2 MDTs
[ 4837.652197] Lustre: DEBUG MARKER: == sanity-quota test 41: df should return projid-specific values ========================================================== 18:15:00 (1755641700)
[ 4846.380410] LustreError: 148582:0:(qsd_reint.c:618:qqi_reint_delayed()) lustre-MDT0000: Delaying reintegration for qtype:0 until pending updates are flushed.
[ 4846.387089] LustreError: 148582:0:(qsd_reint.c:618:qqi_reint_delayed()) Skipped 2 previous similar messages
[ 4887.971620] Lustre: DEBUG MARKER: == sanity-quota test 42: lfs quota should include default quota info ========================================================== 18:15:50 (1755641750)
[ 4916.069855] Lustre: DEBUG MARKER: == sanity-quota test 48: lfs quota --delete should delete quota project ID ========================================================== 18:16:18 (1755641778)
[ 4976.720194] Lustre: DEBUG MARKER: == sanity-quota test 49a: lfs quota -a prints the quota usage for all quota IDs ========================================================== 18:17:19 (1755641839)
[ 4983.873295] Lustre: *** cfs_fail_loc=a09, val=0***
[ 4983.875736] Lustre: Skipped 1 previous similar message
[ 4984.879152] Lustre: *** cfs_fail_loc=a09, val=0***
[ 4984.881558] Lustre: Skipped 99 previous similar messages
[ 4986.883483] Lustre: *** cfs_fail_loc=a09, val=0***
[ 4986.885846] Lustre: Skipped 229 previous similar messages
[ 4990.884816] Lustre: *** cfs_fail_loc=a09, val=0***
[ 4990.887146] Lustre: Skipped 504 previous similar messages
[ 4998.892210] Lustre: *** cfs_fail_loc=a09, val=0***
[ 4998.894751] Lustre: Skipped 924 previous similar messages
[ 5014.893118] Lustre: *** cfs_fail_loc=a09, val=0***
[ 5014.896301] Lustre: Skipped 1838 previous similar messages
[ 5046.909925] Lustre: *** cfs_fail_loc=a09, val=0***
[ 5046.911950] Lustre: Skipped 3376 previous similar messages
[ 5155.097168] LustreError: 148899:0:(qsd_reint.c:618:qqi_reint_delayed()) lustre-MDT0000: Delaying reintegration for qtype:0 until pending updates are flushed.
[ 5155.099913] LustreError: 148899:0:(qsd_reint.c:618:qqi_reint_delayed()) Skipped 4 previous similar messages
[ 5219.131570] LustreError: 143557:0:(qsd_reint.c:618:qqi_reint_delayed()) lustre-MDT0000: Delaying reintegration for qtype:0 until pending updates are flushed.
[ 5219.140195] LustreError: 143557:0:(qsd_reint.c:618:qqi_reint_delayed()) Skipped 208 previous similar messages
[ 5281.248861] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 5281.249335] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 5281.254619] Lustre: Skipped 1 previous similar message
[ 5281.257273] Lustre: Skipped 1 previous similar message
[ 5286.368335] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 5286.371997] Lustre: Skipped 1 previous similar message
[ 5287.274160] Lustre: 154807:0:(qsd_writeback.c:672:qsd_cleanup_deferred()) lustre-MDT0000: Free global deferred upd: ID:4, ver:2047/2044
[ 5287.291491] LustreError: 154807:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup
[ 5287.294772] LustreError: 154807:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages
[ 5287.372747] Lustre: server umount lustre-MDT0000 complete
[ 5289.323108] LustreError: 148680:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1755642152 with bad export cookie 2601557930966505711
[ 5289.328119] LustreError: MGC192.168.204.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 5289.458281] Lustre: server umount lustre-OST0000 complete
[ 5291.596572] Lustre: server umount lustre-OST0001 complete
[ 5294.530782] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_hostid
[ 5298.190573] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing load_modules_local
[ 5302.387113]  vdc: vdc1 vdc9
[ 5306.867862]  vde: vde1 vde9
[ 5311.458336]  vdf: vdf1 vdf9
[ 5318.244193] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing load_modules_local
[ 5321.771176] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000
[ 5321.855047] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space.
[ 5321.889752] Lustre: lustre-MDT0000: new disk, initializing
[ 5322.006070] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 5322.009265] Lustre: Skipped 1 previous similar message
[ 5322.034454] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt
[ 5323.729879] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 5326.585631] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params
[ 5329.137343] Lustre: lustre-OST0000: new disk, initializing
[ 5329.139531] Lustre: srv-lustre-OST0000: No data found on store. Initialize space.
[ 5329.142025] Lustre: Skipped 1 previous similar message
[ 5331.150027] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost
[ 5331.152638] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost]
[ 5331.190736] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400
[ 5331.644265] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 5336.617758] Lustre: lustre-OST0001: new disk, initializing
[ 5336.620950] Lustre: srv-lustre-OST0001: No data found on store. Initialize space.
[ 5337.721217] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost
[ 5337.724388] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost]
[ 5337.769937] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400
[ 5339.491170] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 5345.116227] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 5346.882756] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params
[ 5373.229655] Lustre: DEBUG MARKER: == sanity-quota test 49b: lfs quota -a --blocks has a delimiter ========================================================== 18:23:56 (1755642236)
[ 5376.322536] Lustre: DEBUG MARKER: == sanity-quota test 50: Test if lfs find --projid works ========================================================== 18:23:59 (1755642239)
[ 5408.656499] Lustre: DEBUG MARKER: == sanity-quota test 51: Test project accounting with mv/cp ========================================================== 18:24:31 (1755642271)
[ 5444.269042] Lustre: DEBUG MARKER: == sanity-quota test 52: Rename normal file across project ID ========================================================== 18:25:07 (1755642307)
[ 5458.204856] Lustre: DEBUG MARKER: rename directory return 255
[ 5486.343692] Lustre: DEBUG MARKER: == sanity-quota test 53: Project inherit attribute could be cleared ========================================================== 18:25:49 (1755642349)
[ 5505.045259] Lustre: DEBUG MARKER: == sanity-quota test 54: basic lfs project interface test ========================================================== 18:26:07 (1755642367)
[ 5529.652243] Lustre: DEBUG MARKER: == sanity-quota test 55: Chgrp should be affected by group quota ========================================================== 18:26:32 (1755642392)
[ 5563.231796] LustreError: 166154:0:(qmt_entry.c:552:qmt_adjust_edquot()) $$$ set revoke_time explicitly  qmt:lustre-QMT0000 pool:dt-0x0 id:60001 enforced:1 hard:51200 soft:0 granted:51200 time:0 qunit: 1024 edquot:0 may_rel:0 revoke:0 default:no
[ 5587.578382] Lustre: DEBUG MARKER: == sanity-quota test 56: lfs quota -t should work well === 18:27:30 (1755642450)
[ 5609.246473] Lustre: DEBUG MARKER: == sanity-quota test 57: lfs project could tolerate errors ========================================================== 18:27:52 (1755642472)
[ 5639.364507] Lustre: DEBUG MARKER: == sanity-quota test 58: project ID should be kept for new mirrors created by FID ========================================================== 18:28:22 (1755642502)
[ 5702.837494] LustreError: 158079:0:(qmt_entry.c:552:qmt_adjust_edquot()) $$$ set revoke_time explicitly  qmt:lustre-QMT0000 pool:dt-0x0 id:1001 enforced:1 hard:204800 soft:0 granted:204800 time:0 qunit: 1024 edquot:0 may_rel:0 revoke:0 default:no
[ 5726.505440] Lustre: DEBUG MARKER: == sanity-quota test 59: lfs project dosen't crash kernel with project disabled ========================================================== 18:29:49 (1755642589)
[ 5727.203503] Lustre: DEBUG MARKER: SKIP: sanity-quota test_59 ldiskfs only test
[ 5728.020294] Lustre: DEBUG MARKER: == sanity-quota test 60: Test quota for root with setgid ========================================================== 18:29:50 (1755642590)
[ 5737.130214] LustreError: 171951:0:(qsd_reint.c:618:qqi_reint_delayed()) lustre-MDT0000: Delaying reintegration for qtype:0 until pending updates are flushed.
[ 5737.136910] LustreError: 171951:0:(qsd_reint.c:618:qqi_reint_delayed()) Skipped 2980 previous similar messages
[ 5761.451555] Lustre: DEBUG MARKER: SKIP: sanity-quota test_61 skipping SLOW test 61
[ 5762.239332] Lustre: DEBUG MARKER: == sanity-quota test 62: Project inherit should be only changed by root ========================================================== 18:30:25 (1755642625)
[ 5780.423577] Lustre: DEBUG MARKER: SKIP: sanity-quota test_63 skipping excluded test 63
[ 5781.286525] Lustre: DEBUG MARKER: == sanity-quota test 64: lfs project on non dir/files should succeed ========================================================== 18:30:44 (1755642644)
[ 5813.562705] Lustre: DEBUG MARKER: SKIP: sanity-quota test_65 skipping excluded test 65
[ 5814.371266] Lustre: DEBUG MARKER: == sanity-quota test 66: nonroot user can not change project state in default ========================================================== 18:31:17 (1755642677)
[ 5843.433953] Lustre: DEBUG MARKER: == sanity-quota test 67: quota pools recalculation ======= 18:31:46 (1755642706)
[ 5844.217443] Lustre: DEBUG MARKER: SKIP: sanity-quota test_67 ZFS grants some block space together with inode
[ 5845.123111] Lustre: DEBUG MARKER: == sanity-quota test 68: slave number in quota pool changed after each add/remove OST ========================================================== 18:31:47 (1755642707)
[ 5862.346357] LustreError: 176282:0:(mgs_handler.c:1057:mgs_iocontrol_pool()) OBD_IOC_POOL err -17, cmd CE021 for pool lustre.qpool1
[ 5889.458838] Lustre: DEBUG MARKER: == sanity-quota test 69: EDQUOT at one of pools shouldn't affect DOM ========================================================== 18:32:32 (1755642752)
[ 5905.425290] Lustre: DEBUG MARKER: User quota (block hardlimit:200 MB)
[ 5906.292546] Lustre: DEBUG MARKER: User quota (block hardlimit:10 MB)
[ 5981.544197] Lustre: DEBUG MARKER: == sanity-quota test 70a: check lfs setquota/quota with a pool option ========================================================== 18:34:04 (1755642844)
[ 6012.960977] Lustre: DEBUG MARKER: == sanity-quota test 70b: lfs setquota pool works properly ========================================================== 18:34:35 (1755642875)
[ 6033.540610] Lustre: DEBUG MARKER: == sanity-quota test 71a: Check PFL with quota pools ===== 18:34:56 (1755642896)
[ 6034.318046] Lustre: DEBUG MARKER: SKIP: sanity-quota test_71a ZFS grants some block space together with inode
[ 6035.197468] Lustre: DEBUG MARKER: == sanity-quota test 71b: Check SEL with quota pools ===== 18:34:57 (1755642897)
[ 6035.952646] Lustre: DEBUG MARKER: SKIP: sanity-quota test_71b ZFS grants some block space together with inode
[ 6036.814265] Lustre: DEBUG MARKER: == sanity-quota test 72: lfs quota --pool prints only pool's OSTs ========================================================== 18:34:59 (1755642899)
[ 6046.792821] Lustre: DEBUG MARKER: User quota (block hardlimit:50 MB)
[ 6055.724528] Lustre: DEBUG MARKER: Write...
[ 6056.951222] Lustre: DEBUG MARKER: Write out of block quota ...
[ 6095.124923] Lustre: DEBUG MARKER: == sanity-quota test 73a: default limits at OST Pool Quotas ========================================================== 18:35:57 (1755642957)
[ 6111.113167] Lustre: DEBUG MARKER: set to use default quota
[ 6111.882646] Lustre: DEBUG MARKER: set default quota
[ 6112.607738] Lustre: DEBUG MARKER: get default quota
[ 6114.788032] Lustre: DEBUG MARKER: Test not out of quota
[ 6116.519930] Lustre: DEBUG MARKER: Test out of quota
[ 6120.641423] Lustre: DEBUG MARKER: Increase default quota
[ 6131.974258] Lustre: DEBUG MARKER: Set quota to override default quota
[ 6132.005469] LustreError: 158078:0:(qmt_entry.c:552:qmt_adjust_edquot()) $$$ set revoke_time explicitly  qmt:lustre-QMT0000 pool:dt-qpool1 id:60000 enforced:1 hard:20480 soft:20480 granted:45063 time:1756247795 qunit: 1024 edquot:0 may_rel:0 revoke:0 default:no
[ 6136.679211] Lustre: DEBUG MARKER: Set to use default quota again
[ 6148.441759] Lustre: DEBUG MARKER: Cleanup
[ 6197.997241] Lustre: DEBUG MARKER: == sanity-quota test 73b: default OST Pool Quotas limit for new user ========================================================== 18:37:41 (1755643061)
[ 6213.520275] Lustre: DEBUG MARKER: set default quota for qpool1
[ 6214.165905] Lustre: DEBUG MARKER: Write from user that hasn't lqe
[ 6249.381549] Lustre: DEBUG MARKER: == sanity-quota test 74: check quota pools per user ====== 18:38:32 (1755643112)
[ 6297.591439] Lustre: DEBUG MARKER: == sanity-quota test 75: nodemap squashed root respects quota enforcement ========================================================== 18:39:20 (1755643160)
[ 6386.150837] Lustre: DEBUG MARKER: Write...
[ 6387.835471] Lustre: DEBUG MARKER: Write out of block quota ...
[ 6484.973451] Lustre: DEBUG MARKER: == sanity-quota test 76: project ID 4294967295 should be not allowed ========================================================== 18:42:27 (1755643347)
[ 6514.371582] Lustre: DEBUG MARKER: == sanity-quota test 77: lfs setquota should fail in Lustre mount with 'ro' ========================================================== 18:42:57 (1755643377)
[ 6517.712584] Lustre: DEBUG MARKER: == sanity-quota test 78A: Check fallocate increase quota usage ========================================================== 18:43:00 (1755643380)
[ 6518.468200] Lustre: DEBUG MARKER: SKIP: sanity-quota test_78A need >= 2.13.57 and ldiskfs for fallocate
[ 6519.279795] Lustre: DEBUG MARKER: == sanity-quota test 78a: Check fallocate increase projectid usage ========================================================== 18:43:02 (1755643382)
[ 6519.986290] Lustre: DEBUG MARKER: SKIP: sanity-quota test_78a need >= 2.13.57 and ldiskfs for fallocate
[ 6520.796540] Lustre: DEBUG MARKER: == sanity-quota test 79: access to non-existed dt-pool/info doesn't cause a panic ========================================================== 18:43:03 (1755643383)
[ 6531.985689] Lustre: DEBUG MARKER: == sanity-quota test 80: check for EDQUOT after OST failover ========================================================== 18:43:14 (1755643394)
[ 6532.778716] Lustre: DEBUG MARKER: SKIP: sanity-quota test_80 ZFS grants some block space together with inode
[ 6533.618617] Lustre: DEBUG MARKER: == sanity-quota test 81: Race qmt_start_pool_recalc with qmt_pool_free ========================================================== 18:43:16 (1755643396)
[ 6542.970309] Lustre: DEBUG MARKER: User quota (block hardlimit:20 MB)
[ 6547.391994] LustreError: 198099:0:(qmt_pool.c:1295:qmt_pool_recalc()) cfs_fail_timeout id a07 sleeping for 10000ms
[ 6550.330669] Lustre: lustre-MDT0000: Not available for connect from 192.168.204.50@tcp (stopping)
[ 6551.008712] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 6551.009427] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 6551.015535] Lustre: Skipped 1 previous similar message
[ 6551.021295] Lustre: Skipped 1 previous similar message
[ 6555.446858] Lustre: lustre-MDT0000: Not available for connect from 192.168.204.50@tcp (stopping)
[ 6557.439123] LustreError: 198099:0:(qmt_pool.c:1295:qmt_pool_recalc()) cfs_fail_timeout id a07 awake
[ 6557.531113] LustreError: 198197:0:(obd_class.h:479:obd_check_dev()) Device 17 not setup
[ 6557.533486] LustreError: 198197:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages
[ 6557.623916] Lustre: server umount lustre-MDT0000 complete
[ 6560.536173] LustreError: MGC192.168.204.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 6560.566547] LustreError: 198630:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.204.50@tcp (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 6560.794089] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 6560.797551] Lustre: Skipped 2 previous similar messages
[ 6560.851111] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:27 to 0x240000400:65)
[ 6560.851162] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:27 to 0x280000400:65)
[ 6562.418890] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 6565.858511] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to  (at 0@lo)
[ 6565.862093] Lustre: Skipped 3 previous similar messages
[ 6582.327897] Lustre: DEBUG MARKER: == sanity-quota test 82: verify more than 8 qids for single operation ========================================================== 18:44:05 (1755643445)
[ 6601.879866] Lustre: DEBUG MARKER: == sanity-quota test 83: Setting default quota shouldn't affect grace time ========================================================== 18:44:24 (1755643464)
[ 6620.846489] Lustre: DEBUG MARKER: == sanity-quota test 84: Reset quota should fix the insane granted quota ========================================================== 18:44:43 (1755643483)
[ 6635.867959] LustreError: 158882:0:(qsd_reint.c:618:qqi_reint_delayed()) lustre-OST0000: Delaying reintegration for qtype:0 until pending updates are flushed.
[ 6635.875103] LustreError: 158882:0:(qsd_reint.c:618:qqi_reint_delayed()) Skipped 8 previous similar messages
[ 6648.311587] Lustre: *** cfs_fail_loc=a08, val=0***
[ 6648.314246] Lustre: Skipped 1023 previous similar messages
[ 6648.317556] Lustre: *** cfs_fail_loc=a08, val=0***
[ 6708.863231] Lustre: DEBUG MARKER: == sanity-quota test 85: do not hung at write with the least_qunit ========================================================== 18:46:11 (1755643571)
[ 6765.568555] Lustre: DEBUG MARKER: == sanity-quota test 86: Pre-acquired quota should be released if quota is over limit ========================================================== 18:47:08 (1755643628)
[ 6801.776998] LustreError: 198628:0:(qmt_entry.c:552:qmt_adjust_edquot()) $$$ set revoke_time explicitly  qmt:lustre-QMT0000 pool:md-0x0 id:60000 enforced:1 hard:2048 soft:2048 granted:65536 time:1756248465 qunit: 1024 edquot:0 may_rel:0 revoke:0 default:no
[ 6875.904788] LustreError: 199941:0:(qmt_entry.c:552:qmt_adjust_edquot()) $$$ set revoke_time explicitly  qmt:lustre-QMT0000 pool:md-0x0 id:60000 enforced:1 hard:2048 soft:2048 granted:65536 time:1756248539 qunit: 1024 edquot:0 may_rel:0 revoke:0 default:no
[ 6950.713067] LustreError: 198653:0:(qmt_entry.c:552:qmt_adjust_edquot()) $$$ set revoke_time explicitly  qmt:lustre-QMT0000 pool:md-0x0 id:1000 enforced:1 hard:2048 soft:2048 granted:65536 time:1756248614 qunit: 1024 edquot:0 may_rel:0 revoke:0 default:no
[ 7008.706858] Lustre: DEBUG MARKER: == sanity-quota test 87: lfs quota -a should print default quota setting ========================================================== 18:51:11 (1755643871)
[ 7016.049540] Lustre: *** cfs_fail_loc=a09, val=0***
[ 7052.098629] Lustre: lustre-MDT0000: Not available for connect from 192.168.204.50@tcp (stopping)
[ 7052.103670] Lustre: Skipped 2 previous similar messages
[ 7052.256633] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 7052.263753] Lustre: Skipped 1 previous similar message
[ 7056.556787] LustreError: 206982:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup
[ 7056.559280] LustreError: 206982:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages
[ 7056.630308] Lustre: server umount lustre-MDT0000 complete
[ 7060.211199] LustreError: MGC192.168.204.150@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 7060.485504] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 7060.540252] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:68 to 0x240000400:97)
[ 7060.540310] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:27 to 0x280000400:97)
[ 7062.163984] Lustre: DEBUG MARKER: oleg450-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 7071.717412] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to  (at 0@lo)
[ 7071.721916] Lustre: Skipped 1 previous similar message
[ 7081.175460] Lustre: DEBUG MARKER: == sanity-quota test 88: Writing over quota should not hang ========================================================== 18:52:23 (1755643943)
[ 7081.963468] Lustre: DEBUG MARKER: SKIP: sanity-quota test_88 require client with >4k pages
[ 7082.820803] Lustre: DEBUG MARKER: == sanity-quota test 89: Show default quota with squash_uid ========================================================== 18:52:25 (1755643945)
[ 7123.596294] Lustre: DEBUG MARKER: == sanity-quota test 90a: lfs quota should work without mount point ========================================================== 18:53:06 (1755643986)
[ 7131.388861] Lustre: DEBUG MARKER: sanity-quota test_90a: @@@@@@ FAIL: quota info not Disk quotas for usr root (uid 0):