[ 4356.989658] Lustre: DEBUG MARKER: User quota (limit: 200)
[ 4360.387325] Lustre: DEBUG MARKER: Write 100M (buffered) ...
[ 4362.470815] LustreError: 96652:0:(osd_handler.c:720:osd_ro()) lustre-MDT0000: *** setting device osd-zfs read-only ***
[ 4363.153238] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000
[ 4364.180103] Lustre: DEBUG MARKER: Fail mds for 40 seconds
[ 4365.443480] Lustre: Failing over lustre-MDT0000
[ 4365.662071] LustreError: 96846:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup
[ 4365.664964] LustreError: 96846:0:(obd_class.h:479:obd_check_dev()) Skipped 1 previous similar message
[ 4365.813654] Lustre: server umount lustre-MDT0000 complete
[ 4382.177378] Lustre: 3318:0:(client.c:2468:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1761241761/real 1761241761]  req@ffff8f978737aa00 x1846791353142912/t0(0) o601->lustre-MDT0000-lwp-OST0000@0@lo:23/10 lens 336/336 e 0 to 1 dl 1761241777 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'lquota_wb_lustr.0' uid:0 gid:0 projid:4294967295
[ 4382.199602] Lustre: 3318:0:(client.c:2468:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[ 4382.207732] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 4383.563238] LustreError: MGC192.168.202.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 4384.039094] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 4384.093216] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect
[ 4384.610076] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects
[ 4384.779675] Lustre: lustre-MDT0000: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted.
[ 4384.831664] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:743 to 0x240000400:769)
[ 4384.844662] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:740 to 0x280000400:769)
[ 4385.056203] Lustre: 3319:0:(client.c:2468:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1761241764/real 1761241764]  req@ffff8f9786e95880 x1846791353143168/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1761241780 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
[ 4387.284911] Lustre: DEBUG MARKER: oleg245-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 4388.002640] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo)
[ 4390.176162] Lustre: 3319:0:(client.c:2468:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1761241769/real 1761241769]  req@ffff8f9787379180 x1846791353143552/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1761241785 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
[ 4390.192158] Lustre: 3319:0:(client.c:2468:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[ 4392.852704] Lustre: DEBUG MARKER: oleg245-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 4393.941523] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 4395.488109] Lustre: 3318:0:(client.c:2468:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1761241774/real 1761241774]  req@ffff8f98b9af5500 x1846791353143936/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1761241790 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
[ 4395.508981] Lustre: 3318:0:(client.c:2468:ptlrpc_expire_one_request()) Skipped 2 previous similar messages
[ 4396.837407] LustreError: 81019:0:(qsd_reint.c:618:qqi_reint_delayed()) lustre-OST0000: Delaying reintegration for qtype:0 until pending updates are flushed.
[ 4396.843676] LustreError: 81019:0:(qsd_reint.c:618:qqi_reint_delayed()) Skipped 6 previous similar messages
[ 4409.342366] Lustre: DEBUG MARKER: (dd_pid=93134, time=10, timeout=600)
[ 4444.251113] Lustre: DEBUG MARKER: User quota (limit: 200)
[ 4446.853049] Lustre: DEBUG MARKER: Write 100M (directio) ...
[ 4448.581604] LustreError: 99105:0:(osd_handler.c:720:osd_ro()) lustre-MDT0000: *** setting device osd-zfs read-only ***
[ 4449.199555] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000
[ 4450.427734] Lustre: DEBUG MARKER: Fail mds for 40 seconds
[ 4451.823597] Lustre: Failing over lustre-MDT0000
[ 4452.044070] LustreError: 99299:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup
[ 4452.053240] LustreError: 99299:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages
[ 4452.188500] Lustre: server umount lustre-MDT0000 complete
[ 4468.427986] LustreError: MGC192.168.202.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 4468.985684] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 4469.058827] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect
[ 4469.856130] Lustre: 3319:0:(client.c:2468:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1761241849/real 1761241849]  req@ffff8f98a900f480 x1846791353178496/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1761241865 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
[ 4469.878176] Lustre: 3319:0:(client.c:2468:ptlrpc_expire_one_request()) Skipped 2 previous similar messages
[ 4471.626189] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects
[ 4471.797249] Lustre: lustre-MDT0000: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted.
[ 4471.823936] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:740 to 0x280000400:801)
[ 4471.827271] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:771 to 0x240000400:801)
[ 4472.329845] Lustre: DEBUG MARKER: oleg245-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 4474.737164] LustreError: 81019:0:(qsd_reint.c:618:qqi_reint_delayed()) lustre-OST0000: Delaying reintegration for qtype:0 until pending updates are flushed.
[ 4478.732872] Lustre: DEBUG MARKER: oleg245-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 4480.194281] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 4484.502995] Lustre: DEBUG MARKER: (dd_pid=95336, time=0, timeout=600)
[ 4529.465727] Lustre: DEBUG MARKER: == sanity-quota test 19: Updating admin limits doesn't zero operational limits(b14790) ========================================================== 13:52:03 (1761241923)
[ 4540.424066] LustreError: 101735:0:(qsd_reint.c:618:qqi_reint_delayed()) lustre-MDT0000: Delaying reintegration for qtype:1 until pending updates are flushed.
[ 4546.651487] Lustre: DEBUG MARKER: Files for user (quota_usr), count=1:
[ 4548.040052] Lustre: DEBUG MARKER: Block quota isn't 0 (u:quota_usr:2).
[ 4551.789428] Lustre: DEBUG MARKER: Files for user (quota_usr), count=1:
[ 4553.136493] Lustre: DEBUG MARKER: Block quota isn't 0 (u:quota_usr:2).
[ 4584.068284] Lustre: DEBUG MARKER: == sanity-quota test 20: Test if setquota specifiers work properly (b15754) ========================================================== 13:52:57 (1761241977)
[ 4599.194944] Lustre: DEBUG MARKER: == sanity-quota test 21: Setquota while writing
[ 4611.609170] Lustre: DEBUG MARKER: Set limit(block:10G; file:1000000) for user: quota_usr
[ 4612.807256] Lustre: DEBUG MARKER: Set limit(block:10G; file:1000000) for group: quota_usr
[ 4614.145493] Lustre: DEBUG MARKER: Set limit(block:10G; file:) for project: 1000
[ 4615.531539] Lustre: DEBUG MARKER: Set quota for 1 times
[ 4617.972648] Lustre: DEBUG MARKER: Set quota for 2 times
[ 4620.480379] Lustre: DEBUG MARKER: Set quota for 3 times
[ 4623.256971] Lustre: DEBUG MARKER: Set quota for 4 times
[ 4625.827063] Lustre: DEBUG MARKER: Set quota for 5 times
[ 4628.270524] Lustre: DEBUG MARKER: Set quota for 6 times
[ 4630.747876] Lustre: DEBUG MARKER: Set quota for 7 times
[ 4632.966806] Lustre: DEBUG MARKER: Set quota for 8 times
[ 4635.235357] Lustre: DEBUG MARKER: Set quota for 9 times
[ 4637.470306] Lustre: DEBUG MARKER: Set quota for 10 times
[ 4639.914520] Lustre: DEBUG MARKER: Set quota for 11 times
[ 4642.576486] Lustre: DEBUG MARKER: Set quota for 12 times
[ 4645.148149] Lustre: DEBUG MARKER: Set quota for 13 times
[ 4676.394088] Lustre: DEBUG MARKER: == sanity-quota test 22: enable/disable quota by 'lctl conf_param/set_param -P' ========================================================== 13:54:30 (1761242070)
[ 4678.314910] LustreError: 105495:0:(qsd_reint.c:618:qqi_reint_delayed()) lustre-OST0001: Delaying reintegration for qtype:2 until pending updates are flushed.
[ 4689.377626] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 4689.378602] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 4689.385616] Lustre: Skipped 4 previous similar messages
[ 4692.242395] LustreError: 105884:0:(obd_class.h:479:obd_check_dev()) Device 13 not setup
[ 4692.248383] LustreError: 105884:0:(obd_class.h:479:obd_check_dev()) Skipped 5 previous similar messages
[ 4692.372476] Lustre: server umount lustre-MDT0000 complete
[ 4695.594987] LustreError: 5764:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761242090 with bad export cookie 14541595141673859249
[ 4695.598176] LustreError: MGC192.168.202.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 4695.600437] LustreError: 5764:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message
[ 4695.695037] Lustre: server umount lustre-OST0000 complete
[ 4699.129980] Lustre: server umount lustre-OST0001 complete
[ 4709.948954] Lustre: DEBUG MARKER: oleg245-server.virtnet: executing load_modules_local
[ 4717.275631] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 4720.593308] Lustre: DEBUG MARKER: oleg245-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 4722.937270] Lustre: 107755:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17
[ 4727.326777] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180
[ 4732.253396] Lustre: DEBUG MARKER: oleg245-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 4737.674145] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180
[ 4741.619155] Lustre: DEBUG MARKER: oleg245-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 4741.786483] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:803 to 0x240000400:833)
[ 4741.938494] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:804 to 0x280000400:833)
[ 4747.103925] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 4749.644259] Lustre: 109553:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17
[ 4770.129230] LustreError: 110357:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup
[ 4770.134926] LustreError: 110357:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages
[ 4770.279362] Lustre: server umount lustre-MDT0000 complete
[ 4773.494518] LustreError: 107320:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761242168 with bad export cookie 14541595141673868062
[ 4773.500956] LustreError: 107320:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 1 previous similar message
[ 4773.505450] LustreError: MGC192.168.202.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 4783.172773] Lustre: server umount lustre-OST0000 complete
[ 4797.242244] Lustre: server umount lustre-OST0001 complete
[ 4807.404899] Lustre: DEBUG MARKER: oleg245-server.virtnet: executing load_modules_local
[ 4814.421397] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 4817.562064] Lustre: DEBUG MARKER: oleg245-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 4819.824450] Lustre: 112223:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17
[ 4825.020933] LustreError: 112550:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 4825.052165] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:803 to 0x240000400:865)
[ 4827.945385] Lustre: DEBUG MARKER: oleg245-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 4830.180165] LustreError: 112550:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 4832.948921] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180
[ 4832.952218] Lustre: Skipped 1 previous similar message
[ 4834.378745] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:804 to 0x280000400:865)
[ 4837.182862] Lustre: DEBUG MARKER: oleg245-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 4843.431361] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 4845.863092] Lustre: 114020:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17
[ 4857.986235] Lustre: DEBUG MARKER: == sanity-quota test 23: Quota should be honored with directIO (b16125) ========================================================== 13:57:31 (1761242251)
[ 4859.237486] Lustre: DEBUG MARKER: SKIP: sanity-quota test_23 Overwrite in place is not guaranteed to be space neutral on ZFS
[ 4860.675985] Lustre: DEBUG MARKER: == sanity-quota test 24: lfs draws an asterix when limit is reached (b16646) ========================================================== 13:57:34 (1761242254)
[ 4907.003591] Lustre: DEBUG MARKER: == sanity-quota test 25: check indexes versions ========== 13:58:20 (1761242300)
[ 4947.394437] Lustre: DEBUG MARKER: Write...
[ 4949.669374] Lustre: DEBUG MARKER: Write out of block quota ...
[ 4993.735402] Lustre: DEBUG MARKER: == sanity-quota test 27a: lfs quota/setquota should handle wrong arguments (b19612) ========================================================== 13:59:47 (1761242387)
[ 4998.395934] Lustre: DEBUG MARKER: == sanity-quota test 27b: lfs quota/setquota should handle user/group/project ID (b20200) ========================================================== 13:59:52 (1761242392)
[ 5005.510698] Lustre: DEBUG MARKER: == sanity-quota test 27c: lfs quota should support human-readable output ========================================================== 13:59:59 (1761242399)
[ 5011.460124] Lustre: DEBUG MARKER: == sanity-quota test 27d: lfs setquota should support fraction block limit ========================================================== 14:00:05 (1761242405)
[ 5017.128286] Lustre: DEBUG MARKER: == sanity-quota test 30: Hard limit updates should not reset grace times ========================================================== 14:00:10 (1761242410)
[ 5070.714831] Lustre: DEBUG MARKER: == sanity-quota test 33: Basic usage tracking for user
[ 5172.809984] Lustre: DEBUG MARKER: == sanity-quota test 34: Usage transfer for user
[ 5322.023973] Lustre: DEBUG MARKER: == sanity-quota test 35: Usage is still accessible across reboot ========================================================== 14:05:15 (1761242715)
[ 5364.583943] Lustre: DEBUG MARKER: Restart...
[ 5369.282267] LustreError: 126812:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup
[ 5369.284957] LustreError: 126812:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages
[ 5369.392528] Lustre: server umount lustre-MDT0000 complete
[ 5372.326624] LustreError: 116774:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761242767 with bad export cookie 14541595141673869595
[ 5372.339778] LustreError: MGC192.168.202.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 5382.696580] Lustre: server umount lustre-OST0000 complete
[ 5394.981520] Lustre: server umount lustre-OST0001 complete
[ 5407.055582] Lustre: DEBUG MARKER: oleg245-server.virtnet: executing load_modules_local
[ 5414.611448] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 5417.573507] Lustre: DEBUG MARKER: oleg245-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 5419.846031] Lustre: 128688:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17
[ 5424.801460] LustreError: 129004:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 5424.823200] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:875 to 0x240000400:897)
[ 5427.237791] Lustre: DEBUG MARKER: oleg245-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 5430.246108] LustreError: 129004:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 5433.596532] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:873 to 0x280000400:897)
[ 5435.983897] Lustre: DEBUG MARKER: oleg245-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 5441.483844] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 5444.047713] Lustre: 130471:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17
[ 5489.299974] Lustre: DEBUG MARKER: == sanity-quota test 37: Quota accounted properly for file created by 'lfs setstripe' ========================================================== 14:08:03 (1761242883)
[ 5550.934187] Lustre: DEBUG MARKER: == sanity-quota test 38: Quota accounting iterator doesn't skip id entries ========================================================== 14:09:04 (1761242944)
[ 6969.733570] Lustre: DEBUG MARKER: == sanity-quota test 39: Project ID interface works correctly ========================================================== 14:32:43 (1761244363)
[ 6983.144971] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 6983.148176] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 6983.178716] Lustre: Skipped 1 previous similar message
[ 6983.184858] Lustre: Skipped 2 previous similar messages
[ 6987.427936] LustreError: 135343:0:(obd_class.h:479:obd_check_dev()) Device 9 not setup
[ 6987.432414] LustreError: 135343:0:(obd_class.h:479:obd_check_dev()) Skipped 11 previous similar messages
[ 6987.600700] Lustre: server umount lustre-MDT0000 complete
[ 6991.439445] LustreError: 128260:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1761244386 with bad export cookie 14541595141673876042
[ 6991.444107] LustreError: MGC192.168.202.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 6991.454684] LustreError: 128260:0:(ldlm_lockd.c:2563:ldlm_cancel_handler()) Skipped 2 previous similar messages
[ 6991.575012] Lustre: server umount lustre-OST0000 complete
[ 6995.664785] Lustre: server umount lustre-OST0001 complete
[ 7008.821613] Lustre: DEBUG MARKER: oleg245-server.virtnet: executing load_modules_local
[ 7017.462556] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 7017.464708] Lustre: Skipped 2 previous similar messages
[ 7021.002655] Lustre: DEBUG MARKER: oleg245-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 7023.352345] Lustre: 137199:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17
[ 7028.277044] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180
[ 7033.144940] Lustre: DEBUG MARKER: oleg245-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 7042.292329] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:5899 to 0x280000400:5921)
[ 7042.373876] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:5899 to 0x240000400:5921)
[ 7043.721188] Lustre: DEBUG MARKER: oleg245-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 7050.112419] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 7053.130651] Lustre: 138969:0:(mgs_llog.c:1346:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17
[ 7091.219987] Lustre: DEBUG MARKER: == sanity-quota test 40a: Hard link across different project ID ========================================================== 14:34:44 (1761244484)
[ 7132.859652] Lustre: DEBUG MARKER: == sanity-quota test 40b: Mv across different project ID ========================================================== 14:35:26 (1761244526)
[ 7175.207117] Lustre: DEBUG MARKER: == sanity-quota test 40c: Remote child Dir inherit project quota properly ========================================================== 14:36:08 (1761244568)
[ 7176.610489] Lustre: DEBUG MARKER: SKIP: sanity-quota test_40c needs >= 2 MDTs
[ 7178.402525] Lustre: DEBUG MARKER: == sanity-quota test 40d: Stripe Directory inherit project quota properly ========================================================== 14:36:11 (1761244571)
[ 7179.764083] Lustre: DEBUG MARKER: SKIP: sanity-quota test_40d needs >= 2 MDTs
[ 7181.573338] Lustre: DEBUG MARKER: == sanity-quota test 41: df should return projid-specific values ========================================================== 14:36:14 (1761244574)
[ 7194.768697] LustreError: 141853:0:(qsd_reint.c:618:qqi_reint_delayed()) lustre-MDT0000: Delaying reintegration for qtype:0 until pending updates are flushed.
[ 7194.802222] LustreError: 141853:0:(qsd_reint.c:618:qqi_reint_delayed()) Skipped 2 previous similar messages
[ 7255.014242] Lustre: DEBUG MARKER: == sanity-quota test 42: lfs quota should include default quota info ========================================================== 14:37:28 (1761244648)
[ 7296.402625] Lustre: DEBUG MARKER: == sanity-quota test 48: lfs quota --delete should delete quota project ID ========================================================== 14:38:09 (1761244689)
[ 7414.698892] Lustre: DEBUG MARKER: sanity-quota test_48: @@@@@@ FAIL: QMT: mds1 qid 60000 is not deleted