[ 5376.184529] Lustre: DEBUG MARKER: User quota (limit: 200)
[ 5382.794845] Lustre: DEBUG MARKER: Write 100M (buffered) ...
[ 5386.349732] LustreError: 120757:0:(osd_handler.c:720:osd_ro()) lustre-MDT0000: *** setting device osd-zfs read-only ***
[ 5387.366527] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000
[ 5389.113683] Lustre: DEBUG MARKER: Fail mds for 40 seconds
[ 5391.477120] Lustre: Failing over lustre-MDT0000
[ 5391.986669] Lustre: server umount lustre-MDT0000 complete
[ 5409.760059] Lustre: 3322:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781241367/real 1781241367]  req@ffff93cc8c647480 x1867761357853952/t0(0) o601->lustre-MDT0000-lwp-OST0000@0@lo:23/10 lens 336/336 e 0 to 1 dl 1781241383 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'lquota_wb_lustr.0' uid:0 gid:0 projid:4294967295
[ 5409.808335] Lustre: 3322:0:(client.c:2480:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[ 5409.818522] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 5410.379403] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 5411.219630] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 5411.371891] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect
[ 5411.874699] Lustre: 3323:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781241369/real 1781241369]  req@ffff93cd545b03c0 x1867761357854336/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781241385 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
[ 5411.936290] Lustre: 3323:0:(client.c:2480:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[ 5414.137601] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo)
[ 5416.160175] Lustre: 3320:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781241374/real 1781241374]  req@ffff93cd545b3480 x1867761357854720/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781241390 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
[ 5417.715430] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects
[ 5417.906357] Lustre: lustre-MDT0000: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted.
[ 5417.962755] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:639 to 0x240000400:673)
[ 5417.972423] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:634 to 0x280000400:673)
[ 5418.624952] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 5421.471141] Lustre: 3321:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781241379/real 1781241379]  req@ffff93cd54489680 x1867761357855104/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781241395 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
[ 5421.485905] Lustre: 3321:0:(client.c:2480:ptlrpc_expire_one_request()) Skipped 2 previous similar messages
[ 5425.701686] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 5427.525342] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 5438.780898] Lustre: DEBUG MARKER: (dd_pid=114907, time=5, timeout=600)
[ 5485.003978] Lustre: DEBUG MARKER: User quota (limit: 200)
[ 5491.730100] Lustre: DEBUG MARKER: Write 100M (directio) ...
[ 5494.542967] LustreError: 123500:0:(osd_handler.c:720:osd_ro()) lustre-MDT0000: *** setting device osd-zfs read-only ***
[ 5495.419766] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000
[ 5497.118696] Lustre: DEBUG MARKER: Fail mds for 40 seconds
[ 5499.195879] Lustre: Failing over lustre-MDT0000
[ 5499.648698] LustreError: 121351:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.203.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 5499.692027] LustreError: 121351:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message
[ 5499.793579] Lustre: server umount lustre-MDT0000 complete
[ 5516.767283] Lustre: 3322:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781241474/real 1781241474]  req@ffff93cd73d71e00 x1867761357894016/t0(0) o601->lustre-MDT0000-lwp-OST0000@0@lo:23/10 lens 336/336 e 0 to 1 dl 1781241490 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'lquota_wb_lustr.0' uid:0 gid:0 projid:4294967295
[ 5516.788597] Lustre: 3322:0:(client.c:2480:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[ 5516.794044] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 5516.811222] Lustre: Skipped 1 previous similar message
[ 5517.855468] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 5527.008325] LustreError: 3319:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight  req@ffff93cd767bf840 x1867761357897344/t0(0) o250->MGC192.168.203.116@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
[ 5527.465348] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 5527.602558] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect
[ 5532.200977] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 5539.558917] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 1 client reconnects
[ 5539.727765] Lustre: lustre-MDT0000: Recovery over after 0:01, of 1 clients 1 recovered and 0 were evicted.
[ 5539.783223] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:675 to 0x240000400:705)
[ 5539.783271] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:634 to 0x280000400:705)
[ 5541.665796] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo)
[ 5541.678709] Lustre: Skipped 1 previous similar message
[ 5543.146614] Lustre: DEBUG MARKER: oleg316-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 5544.500954] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 5560.610888] Lustre: DEBUG MARKER: (dd_pid=117314, time=9, timeout=600)
[ 5621.133542] Lustre: DEBUG MARKER: == sanity-quota test 19: Updating admin limits doesn't zero operational limits(b14790) ========================================================== 01:19:53 (1781241593)
[ 5643.945966] Lustre: DEBUG MARKER: Files for user (quota_usr), count=1:
[ 5646.024084] Lustre: DEBUG MARKER: Block quota isn't 0 (u:quota_usr:2).
[ 5652.012226] Lustre: DEBUG MARKER: Files for user (quota_usr), count=1:
[ 5653.654873] Lustre: DEBUG MARKER: Block quota isn't 0 (u:quota_usr:2).
[ 5687.637542] Lustre: DEBUG MARKER: == sanity-quota test 20: Test if setquota specifiers work properly (b15754) ========================================================== 01:20:59 (1781241659)
[ 5713.303964] Lustre: DEBUG MARKER: == sanity-quota test 21: Setquota while writing
[ 5731.308985] Lustre: DEBUG MARKER: Set limit(block:10G; file:1000000) for user: quota_usr
[ 5733.173873] Lustre: DEBUG MARKER: Set limit(block:10G; file:1000000) for group: quota_usr
[ 5734.862727] Lustre: DEBUG MARKER: Set limit(block:10G; file:) for project: 1000
[ 5736.898422] Lustre: DEBUG MARKER: Set quota for 1 times
[ 5740.240585] Lustre: DEBUG MARKER: Set quota for 2 times
[ 5743.295808] Lustre: DEBUG MARKER: Set quota for 3 times
[ 5747.092702] Lustre: DEBUG MARKER: Set quota for 4 times
[ 5750.983705] Lustre: DEBUG MARKER: Set quota for 5 times
[ 5754.805911] Lustre: DEBUG MARKER: Set quota for 6 times
[ 5758.554490] Lustre: DEBUG MARKER: Set quota for 7 times
[ 5762.667436] Lustre: DEBUG MARKER: Set quota for 8 times
[ 5821.945263] Lustre: DEBUG MARKER: == sanity-quota test 22: enable/disable quota by 'lctl conf_param/set_param -P' ========================================================== 01:23:14 (1781241794)
[ 5834.719798] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 5834.752180] Lustre: Skipped 1 previous similar message
[ 5834.761381] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 5838.987747] Lustre: server umount lustre-MDT0000 complete
[ 5843.456995] LustreError: 5770:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781241817 with bad export cookie 8454020124776306725
[ 5843.465391] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 5843.472539] LustreError: 5770:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages
[ 5843.563077] Lustre: server umount lustre-OST0000 complete
[ 5848.055379] Lustre: server umount lustre-OST0001 complete
[ 5862.382872] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local
[ 5871.194705] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 5875.568961] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 5878.244682] Lustre: 133144:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17
[ 5882.892221] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180
[ 5888.223497] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 5894.360779] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180
[ 5896.386033] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:708 to 0x280000400:737)
[ 5896.391909] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:707 to 0x240000400:737)
[ 5900.243841] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 5908.555662] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 5911.928428] Lustre: 134938:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17
[ 5931.238222] Lustre: server umount lustre-MDT0000 complete
[ 5935.535596] LustreError: 132708:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781241909 with bad export cookie 8454020124776313963
[ 5935.549950] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 5945.940309] Lustre: server umount lustre-OST0000 complete
[ 5948.895511] Lustre: 3321:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781241906/real 1781241906]  req@ffff93cc47038f00 x1867761358060416/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781241922 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
[ 5948.922858] Lustre: 3321:0:(client.c:2480:ptlrpc_expire_one_request()) Skipped 9 previous similar messages
[ 5948.929493] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 5948.944476] Lustre: Skipped 1 previous similar message
[ 5949.527462] Lustre: server umount lustre-OST0001 complete
[ 5963.342750] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local
[ 5972.217533] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 5976.458987] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 5979.113670] Lustre: 137559:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17
[ 5990.637348] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 5997.152885] LustreError: 137880:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 5997.203880] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:707 to 0x240000400:769)
[ 5998.486887] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180
[ 5998.490366] Lustre: Skipped 1 previous similar message
[ 6003.724266] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:708 to 0x280000400:769)
[ 6005.383815] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 6014.592635] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 6018.249381] Lustre: 139346:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17
[ 6027.571229] Lustre: DEBUG MARKER: == sanity-quota test 23: Quota should be honored with directIO (b16125) ========================================================== 01:26:39 (1781241999)
[ 6028.983564] Lustre: DEBUG MARKER: SKIP: sanity-quota test_23 Overwrite in place is not guaranteed to be space neutral on ZFS
[ 6030.693868] Lustre: DEBUG MARKER: == sanity-quota test 24: lfs draws an asterix when limit is reached (b16646) ========================================================== 01:26:42 (1781242002)
[ 6086.109538] Lustre: DEBUG MARKER: == sanity-quota test 25: check indexes versions ========== 01:27:38 (1781242058)
[ 6135.459700] Lustre: DEBUG MARKER: Write...
[ 6138.392429] Lustre: DEBUG MARKER: Write out of block quota ...
[ 6192.177027] Lustre: DEBUG MARKER: == sanity-quota test 27a: lfs quota/setquota should handle wrong arguments (b19612) ========================================================== 01:29:24 (1781242164)
[ 6198.942949] Lustre: DEBUG MARKER: == sanity-quota test 27b: lfs quota/setquota should handle user/group/project ID (b20200) ========================================================== 01:29:31 (1781242171)
[ 6208.394305] Lustre: DEBUG MARKER: == sanity-quota test 27c: lfs quota should support human-readable output ========================================================== 01:29:40 (1781242180)
[ 6217.232368] Lustre: DEBUG MARKER: == sanity-quota test 27d: lfs setquota should support fraction block limit ========================================================== 01:29:49 (1781242189)
[ 6225.024784] Lustre: DEBUG MARKER: == sanity-quota test 30: Hard limit updates should not reset grace times ========================================================== 01:29:57 (1781242197)
[ 6285.314448] Lustre: DEBUG MARKER: == sanity-quota test 33: Basic usage tracking for user
[ 6439.456580] Lustre: DEBUG MARKER: == sanity-quota test 34: Usage transfer for user
[ 6619.544406] Lustre: DEBUG MARKER: == sanity-quota test 35: Usage is still accessible across reboot ========================================================== 01:36:31 (1781242591)
[ 6682.981984] Lustre: DEBUG MARKER: Restart...
[ 6687.468357] Lustre: server umount lustre-MDT0000 complete
[ 6691.527498] LustreError: 137132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781242665 with bad export cookie 8454020124776315496
[ 6691.545616] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 6701.181058] Lustre: server umount lustre-OST0000 complete
[ 6714.289709] Lustre: server umount lustre-OST0001 complete
[ 6730.485765] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local
[ 6740.400145] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 6744.762468] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 6747.304423] Lustre: 157947:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17
[ 6752.754469] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180
[ 6759.228307] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 6764.968318] LustreError: 158265:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 6764.994184] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:778 to 0x240000400:801)
[ 6766.794485] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180
[ 6772.229794] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:778 to 0x280000400:801)
[ 6773.383159] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 6780.495446] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 6784.057413] Lustre: 159727:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17
[ 6850.315582] Lustre: DEBUG MARKER: == sanity-quota test 37: Quota accounted properly for file created by 'lfs setstripe' ========================================================== 01:40:22 (1781242822)
[ 6923.399416] Lustre: DEBUG MARKER: == sanity-quota test 38: Quota accounting iterator doesn't skip id entries ========================================================== 01:41:35 (1781242895)
[ 8334.105719] Lustre: DEBUG MARKER: == sanity-quota test 39: Project ID interface works correctly ========================================================== 02:05:05 (1781244305)
[ 8352.739353] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 8352.741255] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 8352.741995] LustreError: lustre-MDT0000-lwp-OST0000: operation quota_acquire to node 0@lo failed: rc = -107
[ 8352.763357] Lustre: Skipped 1 previous similar message
[ 8352.789621] Lustre: Skipped 2 previous similar messages
[ 8353.130944] Lustre: server umount lustre-MDT0000 complete
[ 8357.992954] LustreError: 157516:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781244332 with bad export cookie 8454020124776321992
[ 8358.009976] LustreError: MGC192.168.203.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 8358.013394] LustreError: 157516:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages
[ 8358.124477] Lustre: server umount lustre-OST0000 complete
[ 8362.892373] Lustre: server umount lustre-OST0001 complete
[ 8376.899036] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing load_modules_local
[ 8388.636318] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 8393.593321] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 8396.529198] Lustre: 168264:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17
[ 8401.989753] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180
[ 8408.744169] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 8413.229591] LustreError: 168580:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 8413.268642] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:5804 to 0x240000400:5825)
[ 8416.977535] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180
[ 8422.393797] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000400:5802 to 0x280000400:5825)
[ 8423.661757] Lustre: DEBUG MARKER: oleg316-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 8431.445223] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 8435.453240] Lustre: 170053:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17
[ 8468.583542] Lustre: DEBUG MARKER: == sanity-quota test 40a: Hard link across different project ID ========================================================== 02:07:20 (1781244440)
[ 8513.984699] Lustre: DEBUG MARKER: == sanity-quota test 40b: Mv across different project ID ========================================================== 02:08:06 (1781244486)
[ 8556.672066] Lustre: DEBUG MARKER: == sanity-quota test 40c: Remote child Dir inherit project quota properly ========================================================== 02:08:48 (1781244528)
[ 8558.417986] Lustre: DEBUG MARKER: SKIP: sanity-quota test_40c needs >= 2 MDTs
[ 8560.072841] Lustre: DEBUG MARKER: == sanity-quota test 40d: Stripe Directory inherit project quota properly ========================================================== 02:08:52 (1781244532)
[ 8561.790834] Lustre: DEBUG MARKER: SKIP: sanity-quota test_40d needs >= 2 MDTs
[ 8563.300581] Lustre: DEBUG MARKER: == sanity-quota test 41: df should return projid-specific values ========================================================== 02:08:55 (1781244535)
[ 8653.877363] Lustre: DEBUG MARKER: == sanity-quota test 42: lfs quota should include default quota info ========================================================== 02:10:26 (1781244626)
[ 8702.333171] Lustre: DEBUG MARKER: == sanity-quota test 48: lfs quota --delete should delete quota project ID ========================================================== 02:11:14 (1781244674)
[ 8822.982823] Lustre: DEBUG MARKER: sanity-quota test_48: @@@@@@ FAIL: QMT: mds1 qid 60000 is not deleted