[ 5708.145789] Lustre: DEBUG MARKER: User quota (limit: 200)
[ 5714.388000] Lustre: DEBUG MARKER: Write 100M (buffered) ...
[ 5717.445457] LustreError: 120952:0:(osd_handler.c:720:osd_ro()) lustre-MDT0000: *** setting device osd-zfs read-only ***
[ 5718.308576] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000
[ 5720.187458] Lustre: DEBUG MARKER: Fail mds for 40 seconds
[ 5722.106197] Lustre: Failing over lustre-MDT0000
[ 5722.504889] Lustre: server umount lustre-MDT0000 complete
[ 5738.978360] Lustre: 3312:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1783953471/real 1783953471]  req@ffff8fcd08fb8000 x1870604912385792/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1783953487 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
[ 5739.015724] Lustre: 3312:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[ 5739.022304] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 5739.035029] Lustre: Skipped 1 previous similar message
[ 5739.689251] LustreError: MGC192.168.203.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 5740.177376] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 5740.229298] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect
[ 5741.222206] Lustre: 3315:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1783953473/real 1783953473]  req@ffff8fcc4b369180 x1870604912385920/t0(0) o601->lustre-MDT0000-lwp-OST0000@0@lo:23/10 lens 336/336 e 0 to 1 dl 1783953489 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'lquota_wb_lustr.0' uid:0 gid:0 projid:4294967295
[ 5741.270650] Lustre: 3315:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[ 5744.388266] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 5748.257414] Lustre: 3315:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1783953481/real 1783953481]  req@ffff8fcc15dc0a80 x1870604912386560/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1783953497 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
[ 5748.286620] Lustre: 3315:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 2 previous similar messages
[ 5749.281564] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo)
[ 5749.287070] Lustre: Skipped 1 previous similar message
[ 5749.449615] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects
[ 5749.707390] Lustre: lustre-MDT0000: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted.
[ 5749.790353] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:559 to 0x280000400:577)
[ 5749.802428] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:564 to 0x240000400:609)
[ 5753.951217] Lustre: 3314:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1783953486/real 1783953486]  req@ffff8fcd0304d880 x1870604912387072/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1783953502 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
[ 5753.972502] Lustre: 3314:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[ 5756.086733] Lustre: DEBUG MARKER: oleg345-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
[ 5758.115909] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 5764.303176] Lustre: DEBUG MARKER: (dd_pid=114431, time=0, timeout=600)
[ 5812.776573] Lustre: DEBUG MARKER: User quota (limit: 200)
[ 5818.932358] Lustre: DEBUG MARKER: Write 100M (directio) ...
[ 5822.381354] LustreError: 123819:0:(osd_handler.c:720:osd_ro()) lustre-MDT0000: *** setting device osd-zfs read-only ***
[ 5823.311461] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000
[ 5825.293301] Lustre: DEBUG MARKER: Fail mds for 40 seconds
[ 5827.731733] Lustre: Failing over lustre-MDT0000
[ 5828.076302] LustreError: lustre-MDT0000-lwp-OST0000: operation quota_acquire to node 0@lo failed: rc = -107
[ 5828.082157] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 5828.090093] Lustre: Skipped 1 previous similar message
[ 5828.097828] LustreError: 121558:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 5828.121071] LustreError: 121558:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message
[ 5828.223050] Lustre: server umount lustre-MDT0000 complete
[ 5846.350878] LustreError: MGC192.168.203.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 5846.498180] LustreError: 124420:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 5846.751746] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 5846.806356] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect
[ 5847.655565] Lustre: 3314:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1783953580/real 1783953580]  req@ffff8fcd082a4700 x1870604912425856/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1783953596 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
[ 5847.679184] Lustre: 3314:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[ 5847.720687] LustreError: 3311:0:(ldlm_resource.c:1207:ldlm_resource_complain()) lustre-MDT0000-lwp-OST0001: namespace resource [0x200000006:0x20000:0x0].0x0 (ffff8fcc071b1d00) refcount nonzero (1) after lock cleanup; forcing cleanup.
[ 5847.733510] LustreError: 3311:0:(ldlm_resource.c:1207:ldlm_resource_complain()) Skipped 5 previous similar messages
[ 5847.740566] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo)
[ 5847.749816] Lustre: Skipped 1 previous similar message
[ 5848.047180] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects
[ 5848.180572] Lustre: lustre-MDT0000: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted.
[ 5848.225738] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:611 to 0x240000400:641)
[ 5848.232823] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:559 to 0x280000400:609)
[ 5850.643872] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 5859.427939] Lustre: DEBUG MARKER: oleg345-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0
[ 5861.157455] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 5867.748136] Lustre: DEBUG MARKER: (dd_pid=116873, time=0, timeout=600)
[ 5933.488762] Lustre: DEBUG MARKER: == sanity-quota test 19: Updating admin limits doesn't zero operational limits(b14790) ========================================================== 10:41:20 (1783953680)
[ 5956.871450] Lustre: DEBUG MARKER: Files for user (quota_usr), count=1:
[ 5958.815614] Lustre: DEBUG MARKER: Block quota isn't 0 (u:quota_usr:2).
[ 5966.976600] Lustre: DEBUG MARKER: Files for user (quota_usr), count=1:
[ 5969.296119] Lustre: DEBUG MARKER: Block quota isn't 0 (u:quota_usr:2).
[ 6009.514466] Lustre: DEBUG MARKER: == sanity-quota test 20: Test if setquota specifiers work properly (b15754) ========================================================== 10:42:36 (1783953756)
[ 6036.825505] Lustre: DEBUG MARKER: == sanity-quota test 21: Setquota while writing
[ 6055.815877] Lustre: DEBUG MARKER: Set limit(block:10G; file:1000000) for user: quota_usr
[ 6057.926173] Lustre: DEBUG MARKER: Set limit(block:10G; file:1000000) for group: quota_usr
[ 6059.928308] Lustre: DEBUG MARKER: Set limit(block:10G; file:) for project: 1000
[ 6062.061501] Lustre: DEBUG MARKER: Set quota for 1 times
[ 6066.406651] Lustre: DEBUG MARKER: Set quota for 2 times
[ 6071.056327] Lustre: DEBUG MARKER: Set quota for 3 times
[ 6075.572850] Lustre: DEBUG MARKER: Set quota for 4 times
[ 6079.596922] Lustre: DEBUG MARKER: Set quota for 5 times
[ 6084.591722] Lustre: DEBUG MARKER: Set quota for 6 times
[ 6089.652499] Lustre: DEBUG MARKER: Set quota for 7 times
[ 6132.080788] Lustre: DEBUG MARKER: == sanity-quota test 22: enable/disable quota by 'lctl conf_param/set_param -P' ========================================================== 10:44:38 (1783953878)
[ 6149.088451] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 6149.114199] Lustre: Skipped 1 previous similar message
[ 6149.125274] LustreError: 124418:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 6149.143902] LustreError: 124418:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message
[ 6149.520841] Lustre: server umount lustre-MDT0000 complete
[ 6153.516694] LustreError: 7508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783953902 with bad export cookie 1719225040122282395
[ 6153.526958] LustreError: MGC192.168.203.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 6153.528929] LustreError: 7508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages
[ 6153.611409] Lustre: server umount lustre-OST0000 complete
[ 6157.482521] Lustre: server umount lustre-OST0001 complete
[ 6172.445148] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing load_modules_local
[ 6183.278337] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 6188.900972] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 6192.378271] Lustre: 133393:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17
[ 6198.568888] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180
[ 6205.474353] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 6208.803365] LustreError: 133707:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 6208.835641] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:644 to 0x240000400:673)
[ 6212.620500] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180
[ 6217.714867] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:611 to 0x280000400:641)
[ 6219.052435] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 6226.912109] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 6231.285913] Lustre: 135210:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17
[ 6247.655368] Lustre: server umount lustre-MDT0000 complete
[ 6253.015843] LustreError: 135619:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783954001 with bad export cookie 1719225040122288968
[ 6253.032913] LustreError: MGC192.168.203.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 6262.299408] Lustre: server umount lustre-OST0000 complete
[ 6264.297218] Lustre: 3312:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1783953997/real 1783953997]  req@ffff8fcd35ef1c00 x1870604912584448/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1783954013 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
[ 6264.338209] Lustre: 3312:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 2 previous similar messages
[ 6266.568337] Lustre: server umount lustre-OST0001 complete
[ 6283.849602] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing load_modules_local
[ 6294.971839] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 6299.854536] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 6303.273634] Lustre: 137823:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17
[ 6316.638883] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 6319.136494] LustreError: 138142:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 6319.171240] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:644 to 0x240000400:705)
[ 6325.184793] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180
[ 6325.189153] Lustre: Skipped 1 previous similar message
[ 6330.366214] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:611 to 0x280000400:673)
[ 6332.296648] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 6340.797885] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 6344.929765] Lustre: 139632:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17
[ 6356.889995] Lustre: DEBUG MARKER: == sanity-quota test 23: Quota should be honored with directIO (b16125) ========================================================== 10:48:23 (1783954103)
[ 6358.728111] Lustre: DEBUG MARKER: SKIP: sanity-quota test_23 Overwrite in place is not guaranteed to be space neutral on ZFS
[ 6360.767347] Lustre: DEBUG MARKER: == sanity-quota test 24: lfs draws an asterix when limit is reached (b16646) ========================================================== 10:48:27 (1783954107)
[ 6418.485120] Lustre: DEBUG MARKER: == sanity-quota test 25: check indexes versions ========== 10:49:25 (1783954165)
[ 6473.552506] Lustre: DEBUG MARKER: Write...
[ 6476.203456] Lustre: DEBUG MARKER: Write out of block quota ...
[ 6537.301323] Lustre: DEBUG MARKER: == sanity-quota test 27a: lfs quota/setquota should handle wrong arguments (b19612) ========================================================== 10:51:23 (1783954283)
[ 6543.966539] Lustre: DEBUG MARKER: == sanity-quota test 27b: lfs quota/setquota should handle user/group/project ID (b20200) ========================================================== 10:51:30 (1783954290)
[ 6554.533599] Lustre: DEBUG MARKER: == sanity-quota test 27c: lfs quota should support human-readable output ========================================================== 10:51:41 (1783954301)
[ 6564.393309] Lustre: DEBUG MARKER: == sanity-quota test 27d: lfs setquota should support fraction block limit ========================================================== 10:51:50 (1783954310)
[ 6573.344793] Lustre: DEBUG MARKER: == sanity-quota test 30: Hard limit updates should not reset grace times ========================================================== 10:52:00 (1783954320)
[ 6637.352616] Lustre: DEBUG MARKER: == sanity-quota test 33: Basic usage tracking for user
[ 6797.307238] Lustre: DEBUG MARKER: == sanity-quota test 34: Usage transfer for user
[ 6985.463817] Lustre: DEBUG MARKER: == sanity-quota test 35: Usage is still accessible across reboot ========================================================== 10:58:52 (1783954732)
[ 7051.722653] Lustre: DEBUG MARKER: Restart...
[ 7060.452810] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 7060.458164] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 7060.476316] Lustre: Skipped 3 previous similar messages
[ 7060.499585] Lustre: Skipped 1 previous similar message
[ 7062.496637] LustreError: 3315:0:(client.c:1381:ptlrpc_import_delay_req()) @@@ IMP_CLOSED  req@ffff8fcd11a83480 x1870604912820096/t0(0) o601->lustre-MDT0000-lwp-MDT0000@0@lo:23/10 lens 336/336 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'lquota_wb_lustr.0' uid:0 gid:0 projid:4294967295
[ 7062.517818] LustreError: 156094:0:(lquota_entry.c:135:lqe_iter_cb()) $$$ Inuse quota entry  qsd:lustre-MDT0000 qtype:usr lqe: ffff8fcc060dd800 id:0 enforced:0 granted: 0 pending:0 waiting:0 req:1 usage: 2660 qunit:0 qtune:0 edquot:0 default:no revoke:0
[ 7062.520706] LustreError: 3315:0:(qsd_handler.c:298:qsd_req_completion()) $$$ DQACQ failed with -5, flags:0x8  qsd:lustre-MDT0000 qtype:usr lqe: ffff8fcc060dd800 id:0 enforced:0 granted: 0 pending:0 waiting:0 req:1 usage: 2660 qunit:0 qtune:0 edquot:0 default:no revoke:0
[ 7062.554464] LustreError: 3315:0:(qsd_handler.c:298:qsd_req_completion()) Skipped 2 previous similar messages
[ 7063.750992] Lustre: server umount lustre-MDT0000 complete
[ 7068.702556] LustreError: 137377:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783954817 with bad export cookie 1719225040122290501
[ 7068.704787] LustreError: MGC192.168.203.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 7068.721689] LustreError: 137377:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages
[ 7068.834964] Lustre: server umount lustre-OST0000 complete
[ 7073.823333] Lustre: server umount lustre-OST0001 complete
[ 7091.960799] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing load_modules_local
[ 7102.113799] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 7107.541138] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 7110.887521] Lustre: 157994:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17
[ 7118.691623] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180
[ 7125.962660] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 7126.822476] LustreError: 158318:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 7126.846620] LustreError: 158318:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message
[ 7126.870860] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:715 to 0x240000400:737)
[ 7132.129731] LustreError: 158318:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 7133.333766] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180
[ 7138.812446] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:681 to 0x280000400:705)
[ 7140.032616] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 7148.499742] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 7152.456503] Lustre: 159823:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17
[ 7220.757982] Lustre: DEBUG MARKER: == sanity-quota test 37: Quota accounted properly for file created by 'lfs setstripe' ========================================================== 11:02:47 (1783954967)
[ 7299.434055] Lustre: DEBUG MARKER: == sanity-quota test 38: Quota accounting iterator doesn't skip id entries ========================================================== 11:04:05 (1783955045)
[ 9088.168754] Lustre: DEBUG MARKER: == sanity-quota test 39: Project ID interface works correctly ========================================================== 11:33:54 (1783956834)
[ 9104.864659] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 9104.871362] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 9104.883272] Lustre: Skipped 1 previous similar message
[ 9109.478489] Lustre: server umount lustre-MDT0000 complete
[ 9112.873521] LustreError: 157548:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783956861 with bad export cookie 1719225040122296997
[ 9112.879139] LustreError: MGC192.168.203.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 9112.882601] LustreError: 157548:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message
[ 9112.993569] Lustre: server umount lustre-OST0000 complete
[ 9115.876302] Lustre: server umount lustre-OST0001 complete
[ 9128.579504] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing load_modules_local
[ 9137.629510] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 9141.619113] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 9144.421289] Lustre: 168286:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17
[ 9151.406612] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180
[ 9159.316280] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 9162.670233] LustreError: 168613:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 9162.715834] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:5739 to 0x240000400:5761)
[ 9167.840656] LustreError: 168613:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 9168.565456] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180
[ 9174.037299] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:5707 to 0x280000400:5729)
[ 9175.549611] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 9184.277150] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 9187.790719] Lustre: 170111:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17
[ 9223.653949] Lustre: DEBUG MARKER: == sanity-quota test 40a: Hard link across different project ID ========================================================== 11:36:09 (1783956969)
[ 9269.129435] Lustre: DEBUG MARKER: == sanity-quota test 40b: Mv across different project ID ========================================================== 11:36:55 (1783957015)
[ 9313.797929] Lustre: DEBUG MARKER: == sanity-quota test 40c: Remote child Dir inherit project quota properly ========================================================== 11:37:40 (1783957060)
[ 9315.563516] Lustre: DEBUG MARKER: SKIP: sanity-quota test_40c needs >= 2 MDTs
[ 9317.349222] Lustre: DEBUG MARKER: == sanity-quota test 40d: Stripe Directory inherit project quota properly ========================================================== 11:37:44 (1783957064)
[ 9319.222843] Lustre: DEBUG MARKER: SKIP: sanity-quota test_40d needs >= 2 MDTs
[ 9321.325934] Lustre: DEBUG MARKER: == sanity-quota test 41: df should return projid-specific values ========================================================== 11:37:47 (1783957067)
[ 9411.537950] Lustre: DEBUG MARKER: == sanity-quota test 42: lfs quota should include default quota info ========================================================== 11:39:18 (1783957158)
[ 9461.559882] Lustre: DEBUG MARKER: == sanity-quota test 48: lfs quota --delete should delete quota project ID ========================================================== 11:40:08 (1783957208)
[ 9547.986309] Lustre: DEBUG MARKER: == sanity-quota test 49a: lfs quota -a prints the quota usage for all quota IDs ========================================================== 11:41:34 (1783957294)
[ 9560.382240] Lustre: *** cfs_fail_loc=a09, val=0***
[ 9560.389413] Lustre: Skipped 4 previous similar messages
[ 9562.404839] Lustre: *** cfs_fail_loc=a09, val=0***
[ 9562.406801] Lustre: Skipped 85 previous similar messages
[ 9566.438779] Lustre: *** cfs_fail_loc=a09, val=0***
[ 9566.442447] Lustre: Skipped 175 previous similar messages
[ 9574.467908] Lustre: *** cfs_fail_loc=a09, val=0***
[ 9574.475229] Lustre: Skipped 317 previous similar messages
[ 9590.492507] Lustre: *** cfs_fail_loc=a09, val=0***
[ 9590.499050] Lustre: Skipped 627 previous similar messages
[ 9622.501155] Lustre: *** cfs_fail_loc=a09, val=0***
[ 9622.506474] Lustre: Skipped 1351 previous similar messages
[ 9686.519979] Lustre: *** cfs_fail_loc=a09, val=0***
[ 9686.524019] Lustre: Skipped 2739 previous similar messages
[10103.973827] Lustre: *** cfs_fail_loc=a08, val=0***
[10103.981265] Lustre: Skipped 2699 previous similar messages
[10104.016056] Lustre: *** cfs_fail_loc=a08, val=0***
[10336.236272] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[10336.261193] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[10336.272290] Lustre: Skipped 1 previous similar message
[10336.292712] Lustre: Skipped 2 previous similar messages
[10339.795456] Lustre: server umount lustre-MDT0000 complete
[10344.134360] LustreError: 167840:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1783958093 with bad export cookie 1719225040124076726
[10344.156132] LustreError: MGC192.168.203.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[10344.586548] Lustre: server umount lustre-OST0000 complete
[10349.676214] Lustre: server umount lustre-OST0001 complete
[10358.215629] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_hostid
[10366.522718] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing load_modules_local
[10376.345714]  vdc: vdc1 vdc9
[10385.525204]  vde: vde1 vde9
[10395.105359]  vdf: vdf1 vdf9
[10395.139127]  vdf: vdf1 vdf9
[10395.165718]  vdf: vdf1 vdf9
[10411.996299] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing load_modules_local
[10420.324397] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000
[10420.585177] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space.
[10420.711409] Lustre: lustre-MDT0000: new disk, initializing
[10421.150734] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[10421.292350] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt
[10426.125827] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[10431.389323] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params
[10437.247217] Lustre: lustre-OST0000: new disk, initializing
[10437.256598] Lustre: srv-lustre-OST0000: No data found on store. Initialize space.
[10437.262332] Lustre: Skipped 1 previous similar message
[10437.391095] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180
[10438.399443] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost
[10438.406648] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost]
[10438.569829] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400
[10444.453166] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[10454.267250] Lustre: lustre-OST0001: new disk, initializing
[10454.272613] Lustre: srv-lustre-OST0001: No data found on store. Initialize space.
[10454.385841] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180
[10456.048346] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost
[10456.058881] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost]
[10456.164813] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400
[10460.651706] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[10469.855243] Lustre: DEBUG MARKER: Using TIMEOUT=20
[10473.654319] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params
[10509.736224] Lustre: DEBUG MARKER: == sanity-quota test 49b: lfs quota -a --blocks has a delimiter ========================================================== 11:57:36 (1783958256)
[10514.990789] Lustre: DEBUG MARKER: == sanity-quota test 50: Test if lfs find --projid works ========================================================== 11:57:42 (1783958262)
[10559.153742] Lustre: DEBUG MARKER: == sanity-quota test 51: Test project accounting with mv/cp ========================================================== 11:58:26 (1783958306)
[10620.599372] Lustre: DEBUG MARKER: == sanity-quota test 52: Rename normal file across project ID ========================================================== 11:59:27 (1783958367)
[10654.632781] Lustre: DEBUG MARKER: rename directory return 255
[10702.954664] Lustre: DEBUG MARKER: == sanity-quota test 53: Project inherit attribute could be cleared ========================================================== 12:00:49 (1783958449)
[10737.947876] Lustre: DEBUG MARKER: == sanity-quota test 54: basic lfs project interface test ========================================================== 12:01:24 (1783958484)
[10783.946300] Lustre: DEBUG MARKER: == sanity-quota test 55: Chgrp should be affected by group quota ========================================================== 12:02:10 (1783958530)
[10918.917791] Lustre: DEBUG MARKER: == sanity-quota test 56: lfs quota -t should work well === 12:04:25 (1783958665)
[10959.631363] Lustre: DEBUG MARKER: == sanity-quota test 57: lfs project could tolerate errors ========================================================== 12:05:05 (1783958705)
[11012.470808] Lustre: DEBUG MARKER: == sanity-quota test 58: project ID should be kept for new mirrors created by FID ========================================================== 12:05:58 (1783958758)
[11208.784234] Lustre: DEBUG MARKER: == sanity-quota test 59: lfs project dosen't crash kernel with project disabled ========================================================== 12:09:15 (1783958955)
[11210.495564] Lustre: DEBUG MARKER: SKIP: sanity-quota test_59 ldiskfs only test
[11212.312442] Lustre: DEBUG MARKER: == sanity-quota test 60: Test quota for root with setgid ========================================================== 12:09:18 (1783958958)
[11277.491108] Lustre: DEBUG MARKER: SKIP: sanity-quota test_61 skipping SLOW test 61
[11279.386743] Lustre: DEBUG MARKER: == sanity-quota test 62: Project inherit should be only changed by root ========================================================== 12:10:26 (1783959026)
[11313.401123] Lustre: DEBUG MARKER: SKIP: sanity-quota test_63 skipping excluded test 63
[11315.089209] Lustre: DEBUG MARKER: == sanity-quota test 64: lfs project on non-dir/files should succeed ========================================================== 12:11:01 (1783959061)
[11367.880560] Lustre: DEBUG MARKER: SKIP: sanity-quota test_65 skipping excluded test 65
[11369.550972] Lustre: DEBUG MARKER: == sanity-quota test 66: nonroot user can not change project state in default ========================================================== 12:11:56 (1783959116)
[11418.762912] Lustre: DEBUG MARKER: == sanity-quota test 67: quota pools recalculation ======= 12:12:45 (1783959165)
[11420.232968] Lustre: DEBUG MARKER: SKIP: sanity-quota test_67 ZFS grants some block space together with inode
[11421.818045] Lustre: DEBUG MARKER: == sanity-quota test 68: slave number in quota pool changed after each add/remove OST ========================================================== 12:12:48 (1783959168)
[11450.991576] LustreError: 207869:0:(mgs_handler.c:1144:mgs_iocontrol_pool()) OBD_IOC_POOL err -17, cmd CE021 for pool lustre.qpool1
[11498.487482] Lustre: DEBUG MARKER: == sanity-quota test 69: EDQUOT at one of pools shouldn't affect DOM ========================================================== 12:14:05 (1783959245)
[11523.602076] Lustre: DEBUG MARKER: User quota (block hardlimit:200 MB)
[11525.559281] Lustre: DEBUG MARKER: User quota (block hardlimit:10 MB)
[11717.500887] Lustre: DEBUG MARKER: == sanity-quota test 70a: check lfs setquota/quota with a pool option ========================================================== 12:17:43 (1783959463)
[11770.791629] Lustre: DEBUG MARKER: == sanity-quota test 70b: lfs setquota pool works properly ========================================================== 12:18:37 (1783959517)
[11805.006770] Lustre: DEBUG MARKER: == sanity-quota test 71a: Check PFL with quota pools ===== 12:19:11 (1783959551)
[11806.905403] Lustre: DEBUG MARKER: SKIP: sanity-quota test_71a ZFS grants some block space together with inode
[11809.080087] Lustre: DEBUG MARKER: == sanity-quota test 71b: Check SEL with quota pools ===== 12:19:15 (1783959555)
[11810.549439] Lustre: DEBUG MARKER: SKIP: sanity-quota test_71b ZFS grants some block space together with inode
[11812.518550] Lustre: DEBUG MARKER: == sanity-quota test 72: lfs quota --pool prints only pool's OSTs ========================================================== 12:19:19 (1783959559)
[11830.256576] Lustre: DEBUG MARKER: User quota (block hardlimit:50 MB)
[11847.846740] Lustre: DEBUG MARKER: Write...
[11850.750312] Lustre: DEBUG MARKER: Write out of block quota ...
[11917.470888] Lustre: DEBUG MARKER: == sanity-quota test 73a: default limits at OST Pool Quotas ========================================================== 12:21:04 (1783959664)
[11949.058652] Lustre: DEBUG MARKER: set to use default quota
[11950.973242] Lustre: DEBUG MARKER: set default quota
[11952.733966] Lustre: DEBUG MARKER: get default quota
[11958.975930] Lustre: DEBUG MARKER: Test not out of quota
[11963.292964] Lustre: DEBUG MARKER: Test out of quota
[11975.497447] Lustre: DEBUG MARKER: Increase default quota
[12009.136159] Lustre: DEBUG MARKER: Set quota to override default quota
[12009.224803] LustreError: 186782:0:(qmt_entry.c:558:qmt_adjust_edquot()) $$$ set revoke_time explicitly  qmt:lustre-QMT0000 pool:dt-qpool1 id:60000 enforced:1 hard:20480 soft:20480 granted:45063 time:1784564558 qunit: 1024 edquot:0 may_rel:0 revoke:0 default:no
[12022.718207] Lustre: DEBUG MARKER: Set to use default quota again
[12051.738960] Lustre: DEBUG MARKER: Cleanup
[12126.449976] Lustre: DEBUG MARKER: == sanity-quota test 73b: default OST Pool Quotas limit for new user ========================================================== 12:24:33 (1783959873)
[12154.705556] Lustre: DEBUG MARKER: set default quota for qpool1
[12156.396530] Lustre: DEBUG MARKER: Write from user that hasn't lqe
[12204.888485] Lustre: DEBUG MARKER: == sanity-quota test 74: check quota pools per user ====== 12:25:51 (1783959951)
[12302.566412] Lustre: DEBUG MARKER: == sanity-quota test 75: nodemap squashed root respects quota enforcement ========================================================== 12:27:28 (1783960048)
[12379.912076] Lustre: DEBUG MARKER: Write...
[12383.930078] Lustre: DEBUG MARKER: Write out of block quota ...
[12520.426509] Lustre: DEBUG MARKER: == sanity-quota test 76: project ID 4294967295 should be not allowed ========================================================== 12:31:07 (1783960267)
[12567.692483] Lustre: DEBUG MARKER: == sanity-quota test 77: lfs setquota should fail in Lustre mount with 'ro' ========================================================== 12:31:54 (1783960314)
[12574.743676] Lustre: DEBUG MARKER: == sanity-quota test 78A: Check fallocate increase quota usage ========================================================== 12:32:01 (1783960321)
[12576.447734] Lustre: DEBUG MARKER: SKIP: sanity-quota test_78A need >= 2.13.57 and ldiskfs for fallocate
[12578.455457] Lustre: DEBUG MARKER: == sanity-quota test 78a: Check fallocate increase projectid usage ========================================================== 12:32:04 (1783960324)
[12580.729531] Lustre: DEBUG MARKER: SKIP: sanity-quota test_78a need >= 2.13.57 and ldiskfs for fallocate
[12584.166433] Lustre: DEBUG MARKER: == sanity-quota test 79: access to non-existed dt-pool/info doesn't cause a panic ========================================================== 12:32:09 (1783960329)
[12609.814476] Lustre: DEBUG MARKER: == sanity-quota test 80: check for EDQUOT after OST failover ========================================================== 12:32:36 (1783960356)
[12611.939713] Lustre: DEBUG MARKER: SKIP: sanity-quota test_80 ZFS grants some block space together with inode
[12613.629986] Lustre: DEBUG MARKER: == sanity-quota test 81: Race qmt_start_pool_recalc with qmt_pool_free ========================================================== 12:32:40 (1783960360)
[12631.377276] Lustre: DEBUG MARKER: User quota (block hardlimit:20 MB)
[12638.324750] LustreError: 232730:0:(qmt_pool.c:1413:qmt_pool_recalc()) cfs_fail_timeout id a07 sleeping for 10000ms
[12642.272112] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[12642.276159] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[12642.295521] Lustre: Skipped 1 previous similar message
[12642.318839] Lustre: Skipped 1 previous similar message
[12645.094669] Lustre: lustre-MDT0000: Not available for connect from 192.168.203.45@tcp (stopping)
[12647.392512] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[12647.407288] Lustre: Skipped 1 previous similar message
[12648.335630] LustreError: 232730:0:(qmt_pool.c:1413:qmt_pool_recalc()) cfs_fail_timeout id a07 awake
[12648.633332] Lustre: server umount lustre-MDT0000 complete
[12656.556082] LustreError: MGC192.168.203.145@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[12657.184737] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[12657.299212] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:27 to 0x280000400:65)
[12657.308874] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:27 to 0x240000400:65)
[12661.409543] Lustre: DEBUG MARKER: oleg345-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[12672.485773] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo)
[12672.495944] Lustre: Skipped 1 previous similar message
[12696.879407] Lustre: DEBUG MARKER: == sanity-quota test 82: verify more than 8 qids for single operation ========================================================== 12:34:03 (1783960443)
[12730.128111] Lustre: DEBUG MARKER: == sanity-quota test 83: Setting default quota shouldn't affect grace time ========================================================== 12:34:36 (1783960476)
[12761.583795] Lustre: DEBUG MARKER: == sanity-quota test 84: Reset quota should fix the insane granted quota ========================================================== 12:35:08 (1783960508)
[12812.489240] Lustre: *** cfs_fail_loc=a08, val=0***
[12812.491860] Lustre: Skipped 15 previous similar messages
[12812.503148] Lustre: *** cfs_fail_loc=a08, val=0***
[12812.506877] Lustre: Skipped 15 previous similar messages
[12934.024679] Lustre: DEBUG MARKER: == sanity-quota test 85: do not hung at write with the least_qunit ========================================================== 12:38:00 (1783960680)
[13016.269071] Lustre: DEBUG MARKER: == sanity-quota test 86: Pre-acquired quota should be released if quota is over limit ========================================================== 12:39:22 (1783960762)
[13101.068423] LustreError: 233270:0:(qmt_entry.c:558:qmt_adjust_edquot()) $$$ set revoke_time explicitly  qmt:lustre-QMT0000 pool:md-0x0 id:60000 enforced:1 hard:2048 soft:2048 granted:65536 time:1784565650 qunit: 1024 edquot:0 may_rel:0 revoke:0 default:no
[13283.537872] LustreError: 234400:0:(qmt_entry.c:558:qmt_adjust_edquot()) $$$ set revoke_time explicitly  qmt:lustre-QMT0000 pool:md-0x0 id:60000 enforced:1 hard:2048 soft:2048 granted:65536 time:1784565832 qunit: 1024 edquot:0 may_rel:0 revoke:0 default:no
[13452.702892] LustreError: 233271:0:(qmt_entry.c:558:qmt_adjust_edquot()) $$$ set revoke_time explicitly  qmt:lustre-QMT0000 pool:md-0x0 id:1000 enforced:1 hard:2048 soft:2048 granted:65536 time:1784566001 qunit: 1024 edquot:0 may_rel:0 revoke:0 default:no
[13461.676750] Lustre: DEBUG MARKER: sanity-quota test_86: @@@@@@ FAIL: succeed to create files, expect failed