[ 2043.667713] Lustre: DEBUG MARKER: User quota (limit: 200)
[ 2044.757001] Lustre: DEBUG MARKER: Write 100M (buffered) ...
[ 2046.749659] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000
[ 2047.114677] Lustre: DEBUG MARKER: Fail mds for 40 seconds
[ 2047.584730] Lustre: Failing over lustre-MDT0000
[ 2047.658040] LustreError: 26585:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup
[ 2047.660889] LustreError: 26585:0:(obd_class.h:479:obd_check_dev()) Skipped 3 previous similar messages
[ 2047.685378] Lustre: server umount lustre-MDT0000 complete
[ 2049.152092] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107
[ 2049.156395] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 2049.163265] LustreError: 30869:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 2049.168456] LustreError: 30869:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 2 previous similar messages
[ 2060.590221] LDISKFS-fs (dm-0): recovery complete
[ 2060.591743] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc
[ 2060.616283] LustreError: MGC192.168.203.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 2060.673211] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 2060.680836] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect
[ 2060.984080] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 2 clients reconnect
[ 2061.428294] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 2065.681144] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to  (at 0@lo)
[ 2065.682941] Lustre: Skipped 1 previous similar message
[ 2065.687940] Lustre: lustre-MDT0000: Recovery over after 0:05, of 2 clients 2 recovered and 0 were evicted.
[ 2065.702806] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:1798 to 0x2c0000401:1825)
[ 2065.702922] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:1802 to 0x280000401:1825)
[ 2066.451494] Lustre: DEBUG MARKER: oleg334-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 2066.826715] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 2068.890356] Lustre: DEBUG MARKER: (dd_pid=16253, time=0, timeout=600)
[ 2075.105815] Lustre: DEBUG MARKER: User quota (limit: 200)
[ 2076.152207] Lustre: DEBUG MARKER: Write 100M (directio) ...
[ 2078.007059] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000
[ 2078.340036] Lustre: DEBUG MARKER: Fail mds for 40 seconds
[ 2078.815059] Lustre: Failing over lustre-MDT0000
[ 2078.902226] Lustre: server umount lustre-MDT0000 complete
[ 2085.712057] LustreError: 30868:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 2085.717480] LustreError: 30868:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 18 previous similar messages
[ 2091.803707] LDISKFS-fs (dm-0): recovery complete
[ 2091.805297] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc
[ 2091.830049] LustreError: MGC192.168.203.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 2091.893570] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 2092.642775] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 2096.918703] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:1827 to 0x280000401:1857)
[ 2096.918806] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:1798 to 0x2c0000401:1857)
[ 2097.722689] Lustre: DEBUG MARKER: oleg334-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 2098.090584] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 2100.130826] Lustre: DEBUG MARKER: (dd_pid=18547, time=0, timeout=600)
[ 2109.279345] Lustre: DEBUG MARKER: == sanity-quota test 19: Updating admin limits doesn't zero operational limits(b14790) ========================================================== 06:19:23 (1744885163)
[ 2114.128777] LustreError: 21092:0:(qsd_reint.c:617:qqi_reint_delayed()) lustre-OST0001: Delaying reintegration for qtype:0 until pending updates are flushed.
[ 2114.131985] LustreError: 21092:0:(qsd_reint.c:617:qqi_reint_delayed()) Skipped 7 previous similar messages
[ 2124.913419] Lustre: DEBUG MARKER: == sanity-quota test 20: Test if setquota specifiers work properly (b15754) ========================================================== 06:19:38 (1744885178)
[ 2130.814320] Lustre: DEBUG MARKER: == sanity-quota test 21: Setquota while writing
[ 2133.147566] Lustre: DEBUG MARKER: Set limit(block:10G; file:1000000) for user: quota_usr
[ 2133.483482] Lustre: DEBUG MARKER: Set limit(block:10G; file:1000000) for group: quota_usr
[ 2133.824999] Lustre: DEBUG MARKER: Set limit(block:10G; file:) for project: 1000
[ 2134.189222] Lustre: DEBUG MARKER: Set quota for 1 times
[ 2135.634233] Lustre: DEBUG MARKER: Set quota for 2 times
[ 2137.058240] Lustre: DEBUG MARKER: Set quota for 3 times
[ 2138.454906] Lustre: DEBUG MARKER: Set quota for 4 times
[ 2139.882172] Lustre: DEBUG MARKER: Set quota for 5 times
[ 2141.306152] Lustre: DEBUG MARKER: Set quota for 6 times
[ 2142.721158] Lustre: DEBUG MARKER: Set quota for 7 times
[ 2144.112311] Lustre: DEBUG MARKER: Set quota for 8 times
[ 2145.508157] Lustre: DEBUG MARKER: Set quota for 9 times
[ 2146.924430] Lustre: DEBUG MARKER: Set quota for 10 times
[ 2148.333124] Lustre: DEBUG MARKER: Set quota for 11 times
[ 2149.733417] Lustre: DEBUG MARKER: Set quota for 12 times
[ 2151.137968] Lustre: DEBUG MARKER: Set quota for 13 times
[ 2152.561636] Lustre: DEBUG MARKER: Set quota for 14 times
[ 2153.987734] Lustre: DEBUG MARKER: Set quota for 15 times
[ 2155.366167] Lustre: DEBUG MARKER: Set quota for 16 times
[ 2156.790978] Lustre: DEBUG MARKER: Set quota for 17 times
[ 2158.173231] Lustre: DEBUG MARKER: Set quota for 18 times
[ 2159.558691] Lustre: DEBUG MARKER: Set quota for 19 times
[ 2160.935845] Lustre: DEBUG MARKER: Set quota for 20 times
[ 2162.344806] Lustre: DEBUG MARKER: Set quota for 21 times
[ 2163.761973] Lustre: DEBUG MARKER: Set quota for 22 times
[ 2180.859534] Lustre: DEBUG MARKER: == sanity-quota test 22: enable/disable quota by 'lctl conf_param/set_param -P' ========================================================== 06:20:34 (1744885234)
[ 2187.041236] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 2187.044312] Lustre: Skipped 3 previous similar messages
[ 2191.688663] LustreError: 7253:0:(obd_class.h:479:obd_check_dev()) Device 22 not setup
[ 2191.690496] LustreError: 7253:0:(obd_class.h:479:obd_check_dev()) Skipped 15 previous similar messages
[ 2191.711409] Lustre: server umount lustre-MDT0000 complete
[ 2192.048568] LustreError: 8757:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 2192.059864] LustreError: 8757:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 9 previous similar messages
[ 2192.997404] LustreError: 8361:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744885248 with bad export cookie 16140171016234908611
[ 2192.997993] LustreError: MGC192.168.203.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 2193.012076] LustreError: 8361:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) Skipped 2 previous similar messages
[ 2193.147574] Lustre: server umount lustre-MDT0001 complete
[ 2205.079073] Lustre: server umount lustre-OST0000 complete
[ 2216.076343] Lustre: server umount lustre-OST0001 complete
[ 2219.409246] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing load_modules_local
[ 2222.157117] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc
[ 2222.346919] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 2223.220479] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 2225.886547] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 2225.930220] Lustre: 10336:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.203.134@tcp: no remote llog for lustre-sptlrpc, check MGS config
[ 2226.910208] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 2227.734005] Lustre: Modifying parameter general.debug_raw_pointers in log params
[ 2229.528833] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5
[ 2229.532383] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 2229.644424] mount.lustre (11272) used greatest stack depth: 9328 bytes left
[ 2230.904876] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 2232.631673] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:1859 to 0x280000401:1889)
[ 2233.107081] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5
[ 2233.110747] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 2234.398399] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 2234.928710] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:1860 to 0x2c0000401:1889)
[ 2235.928827] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:97)
[ 2240.621111] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 2241.575493] Lustre: Modifying parameter general.lod.*.mdt_hash in log params
[ 2253.865045] Lustre: server umount lustre-MDT0000 complete
[ 2254.777070] LustreError: 13864:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744885310 with bad export cookie 16140171016234933818
[ 2254.778082] LustreError: MGC192.168.203.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 2254.782965] LustreError: 13864:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) Skipped 3 previous similar messages
[ 2255.952318] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping)
[ 2255.956638] Lustre: Skipped 1 previous similar message
[ 2259.960102] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping)
[ 2259.962667] Lustre: Skipped 1 previous similar message
[ 2260.880147] Lustre: server umount lustre-MDT0001 complete
[ 2261.912666] Lustre: server umount lustre-OST0000 complete
[ 2262.890247] Lustre: server umount lustre-OST0001 complete
[ 2265.900706] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing load_modules_local
[ 2268.148348] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc
[ 2268.290248] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 2268.292961] Lustre: Skipped 3 previous similar messages
[ 2269.045171] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 2271.132877] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 2271.157457] Lustre: 17313:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.203.134@tcp: no remote llog for lustre-sptlrpc, check MGS config
[ 2271.160977] Lustre: 17313:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 2 previous similar messages
[ 2272.018164] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 2272.821582] Lustre: Modifying parameter general.debug_raw_pointers in log params
[ 2274.908958] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5
[ 2274.917374] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 2276.584135] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 2279.056405] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:1859 to 0x280000401:1921)
[ 2279.356415] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5
[ 2279.360374] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 2280.440659] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:1860 to 0x2c0000401:1921)
[ 2280.444139] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:129)
[ 2280.925073] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 2283.776455] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 2284.768417] Lustre: Modifying parameter general.lod.*.mdt_hash in log params
[ 2292.511380] Lustre: DEBUG MARKER: == sanity-quota test 23: Quota should be honored with directIO (b16125) ========================================================== 06:22:26 (1744885346)
[ 2292.918201] Lustre: DEBUG MARKER: OST0_SIZE: 3605408 required: 6144
[ 2293.271801] Lustre: DEBUG MARKER: run for 4MB test file
[ 2297.337077] Lustre: DEBUG MARKER: User quota (limit: 4 MB)
[ 2298.483352] Lustre: DEBUG MARKER: Step1: trigger EDQUOT with O_DIRECT
[ 2298.838169] Lustre: DEBUG MARKER: Write half of file
[ 2299.244105] Lustre: DEBUG MARKER: Write out of block quota ...
[ 2299.641811] Lustre: DEBUG MARKER: Step1: done
[ 2300.009034] Lustre: DEBUG MARKER: Step2: rewrite should succeed
[ 2300.407355] Lustre: DEBUG MARKER: Step2: done
[ 2312.902152] Lustre: DEBUG MARKER: OST0_SIZE: 3605408 required: 61440
[ 2313.354944] Lustre: DEBUG MARKER: run for 40MB test file
[ 2316.516144] Lustre: DEBUG MARKER: User quota (limit: 40 MB)
[ 2317.823051] Lustre: DEBUG MARKER: Step1: trigger EDQUOT with O_DIRECT
[ 2318.268390] Lustre: DEBUG MARKER: Write half of file
[ 2319.078685] Lustre: DEBUG MARKER: Write out of block quota ...
[ 2319.874943] Lustre: DEBUG MARKER: Step1: done
[ 2320.266311] Lustre: DEBUG MARKER: Step2: rewrite should succeed
[ 2320.733713] Lustre: DEBUG MARKER: Step2: done
[ 2339.532159] Lustre: DEBUG MARKER: == sanity-quota test 24: lfs draws an asterix when limit is reached (b16646) ========================================================== 06:23:13 (1744885393)
[ 2353.776815] Lustre: DEBUG MARKER: == sanity-quota test 25: check indexes versions ========== 06:23:27 (1744885407)
[ 2365.764577] Lustre: DEBUG MARKER: Write...
[ 2366.278628] Lustre: DEBUG MARKER: Write out of block quota ...
[ 2389.304819] Lustre: DEBUG MARKER: == sanity-quota test 27a: lfs quota/setquota should handle wrong arguments (b19612) ========================================================== 06:24:03 (1744885443)
[ 2390.940035] Lustre: DEBUG MARKER: == sanity-quota test 27b: lfs quota/setquota should handle user/group/project ID (b20200) ========================================================== 06:24:04 (1744885444)
[ 2394.714977] Lustre: DEBUG MARKER: == sanity-quota test 27c: lfs quota should support human-readable output ========================================================== 06:24:08 (1744885448)
[ 2397.457687] Lustre: DEBUG MARKER: == sanity-quota test 27d: lfs setquota should support fraction block limit ========================================================== 06:24:11 (1744885451)
[ 2400.133823] Lustre: DEBUG MARKER: == sanity-quota test 30: Hard limit updates should not reset grace times ========================================================== 06:24:13 (1744885453)
[ 2424.712902] Lustre: DEBUG MARKER: == sanity-quota test 33: Basic usage tracking for user
[ 2461.591598] Lustre: DEBUG MARKER: == sanity-quota test 34: Usage transfer for user
[ 2518.698463] Lustre: DEBUG MARKER: == sanity-quota test 35: Usage is still accessible across reboot ========================================================== 06:26:12 (1744885572)
[ 2532.274726] Lustre: DEBUG MARKER: Restart...
[ 2535.840233] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 2535.840485] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 2535.846839] Lustre: Skipped 17 previous similar messages
[ 2538.945676] LustreError: 11645:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup
[ 2538.947711] LustreError: 11645:0:(obd_class.h:479:obd_check_dev()) Skipped 51 previous similar messages
[ 2538.963871] Lustre: server umount lustre-MDT0000 complete
[ 2539.857421] LustreError: 16434:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744885595 with bad export cookie 16140171016234936121
[ 2539.858469] LustreError: MGC192.168.203.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 2539.865568] LustreError: 16434:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) Skipped 2 previous similar messages
[ 2539.967411] Lustre: server umount lustre-MDT0001 complete
[ 2550.952271] Lustre: server umount lustre-OST0000 complete
[ 2561.976794] Lustre: server umount lustre-OST0001 complete
[ 2565.451419] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing load_modules_local
[ 2567.897636] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc
[ 2568.024667] LustreError: 13852:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 2568.033365] LustreError: 13852:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 8 previous similar messages
[ 2568.051321] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 2568.054303] Lustre: Skipped 3 previous similar messages
[ 2568.857511] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 2571.100941] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 2571.129708] Lustre: 14716:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.203.134@tcp: no remote llog for lustre-sptlrpc, check MGS config
[ 2571.134171] Lustre: 14716:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 2 previous similar messages
[ 2572.044547] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 2572.843132] Lustre: Modifying parameter general.debug_raw_pointers in log params
[ 2574.654458] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5
[ 2574.658288] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 2576.177960] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 2577.767267] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:1932 to 0x280000401:1953)
[ 2578.531811] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5
[ 2578.539585] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 2579.973030] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 2580.590255] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:1930 to 0x2c0000401:1953)
[ 2581.590661] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:161)
[ 2586.242566] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 2609.157770] Lustre: DEBUG MARKER: == sanity-quota test 37: Quota accounted properly for file created by 'lfs setstripe' ========================================================== 06:27:43 (1744885663)
[ 2630.361228] Lustre: DEBUG MARKER: == sanity-quota test 38: Quota accounting iterator doesn't skip id entries ========================================================== 06:28:04 (1744885684)
[ 2940.400183] Lustre: DEBUG MARKER: == sanity-quota test 39: Project ID interface works correctly ========================================================== 06:33:14 (1744885994)
[ 2942.160620] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 2942.162957] Lustre: Skipped 5 previous similar messages
[ 2946.800233] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107
[ 2948.124617] LustreError: 23643:0:(obd_class.h:479:obd_check_dev()) Device 10 not setup
[ 2948.126763] LustreError: 23643:0:(obd_class.h:479:obd_check_dev()) Skipped 25 previous similar messages
[ 2948.170514] Lustre: server umount lustre-MDT0000 complete
[ 2949.159424] LustreError: 15667:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744886004 with bad export cookie 16140171016234944955
[ 2949.160656] LustreError: MGC192.168.203.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 2949.169820] LustreError: 15667:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) Skipped 4 previous similar messages
[ 2949.278126] Lustre: server umount lustre-MDT0001 complete
[ 2961.205764] Lustre: server umount lustre-OST0000 complete
[ 2972.180615] Lustre: server umount lustre-OST0001 complete
[ 2975.291818] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing load_modules_local
[ 2977.487489] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc
[ 2977.603407] LustreError: 25850:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 2977.610393] LustreError: 25850:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 1 previous similar message
[ 2977.625115] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 2977.627731] Lustre: Skipped 3 previous similar messages
[ 2978.422014] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 2980.630424] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 2980.657939] Lustre: 26712:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.203.134@tcp: no remote llog for lustre-sptlrpc, check MGS config
[ 2980.663141] Lustre: 26712:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 2 previous similar messages
[ 2981.570905] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 2982.355583] Lustre: Modifying parameter general.debug_raw_pointers in log params
[ 2982.357611] Lustre: Skipped 1 previous similar message
[ 2984.095623] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5
[ 2984.101200] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 2985.507463] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 2987.725623] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5
[ 2987.730210] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 2988.782779] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:6956 to 0x280000401:6977)
[ 2988.784725] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:6954 to 0x2c0000401:6977)
[ 2989.033125] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 2990.782980] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:193)
[ 2995.312505] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 3012.073889] Lustre: DEBUG MARKER: == sanity-quota test 40a: Hard link across different project ID ========================================================== 06:34:25 (1744886065)
[ 3021.447565] Lustre: DEBUG MARKER: == sanity-quota test 40b: Mv across different project ID ========================================================== 06:34:35 (1744886075)
[ 3032.144942] Lustre: DEBUG MARKER: == sanity-quota test 40c: Remote child Dir inherit project quota properly ========================================================== 06:34:46 (1744886086)
[ 3043.349539] Lustre: DEBUG MARKER: == sanity-quota test 40d: Stripe Directory inherit project quota properly ========================================================== 06:34:57 (1744886097)
[ 3053.562464] Lustre: DEBUG MARKER: == sanity-quota test 41: df should return projid-specific values ========================================================== 06:35:07 (1744886107)
[ 3056.790272] LustreError: 3075:0:(qsd_reint.c:617:qqi_reint_delayed()) lustre-MDT0000: Delaying reintegration for qtype:0 until pending updates are flushed.
[ 3056.795352] LustreError: 3075:0:(qsd_reint.c:617:qqi_reint_delayed()) Skipped 15 previous similar messages
[ 3076.192773] Lustre: DEBUG MARKER: == sanity-quota test 42: lfs quota should include default quota info ========================================================== 06:35:30 (1744886130)
[ 3083.859640] Lustre: DEBUG MARKER: == sanity-quota test 48: lfs quota --delete should delete quota project ID ========================================================== 06:35:37 (1744886137)
[ 3120.584452] Lustre: DEBUG MARKER: == sanity-quota test 49: lfs quota -a prints the quota usage for all quota IDs ========================================================== 06:36:14 (1744886174)
[ 3121.773383] Lustre: *** cfs_fail_loc=a09, val=0***
[ 3121.774807] Lustre: Skipped 1 previous similar message
[ 3122.778056] Lustre: *** cfs_fail_loc=a09, val=0***
[ 3122.779424] Lustre: Skipped 257 previous similar messages
[ 3124.786937] Lustre: *** cfs_fail_loc=a09, val=0***
[ 3124.789170] Lustre: Skipped 495 previous similar messages
[ 3128.792325] Lustre: *** cfs_fail_loc=a09, val=0***
[ 3128.793308] Lustre: Skipped 1073 previous similar messages
[ 3136.796560] Lustre: *** cfs_fail_loc=a09, val=0***
[ 3136.798666] Lustre: Skipped 2163 previous similar messages
[ 3236.144108] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107
[ 3236.147018] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete
[ 3236.150597] Lustre: Skipped 5 previous similar messages
[ 3236.152235] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 3236.154733] Lustre: Skipped 4 previous similar messages
[ 3241.879787] Lustre: server umount lustre-MDT0000 complete
[ 3242.888348] LustreError: 25832:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744886298 with bad export cookie 16140171016236676874
[ 3242.890014] LustreError: MGC192.168.203.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 3242.897484] LustreError: 25832:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) Skipped 4 previous similar messages
[ 3249.008259] Lustre: server umount lustre-MDT0001 complete
[ 3250.176728] Lustre: server umount lustre-OST0000 complete
[ 3251.395383] Lustre: server umount lustre-OST0001 complete
[ 3252.853764] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_hostid
[ 3254.370230] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing load_modules_local
[ 3259.581292] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro
[ 3263.740507] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro
[ 3265.689002] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5
[ 3265.693587] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro
[ 3267.629495] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5
[ 3267.632868] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro
[ 3270.408128] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing load_modules_local
[ 3272.666633] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: errors=remount-ro
[ 3272.686779] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc
[ 3272.760498] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall in log lustre-MDT0000
[ 3272.764055] Lustre: Skipped 1 previous similar message
[ 3272.771782] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space.
[ 3272.808046] Lustre: lustre-MDT0000: new disk, initializing
[ 3272.824432] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 3272.826324] Lustre: Skipped 3 previous similar messages
[ 3272.830704] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt
[ 3273.597367] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3277.004016] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro
[ 3277.027952] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 3277.055117] Lustre: 15934:0:(mgc_request_server.c:550:mgc_llog_local_copy()) MGC192.168.203.134@tcp: no remote llog for lustre-sptlrpc, check MGS config
[ 3277.057884] Lustre: 15934:0:(mgc_request_server.c:550:mgc_llog_local_copy()) Skipped 2 previous similar messages
[ 3277.062922] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space.
[ 3277.065029] Lustre: Skipped 1 previous similar message
[ 3277.101598] Lustre: lustre-MDT0001: new disk, initializing
[ 3277.120089] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt
[ 3277.123348] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt]
[ 3277.864626] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3281.557935] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5
[ 3281.560912] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro
[ 3281.581359] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5
[ 3281.585619] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 3281.657936] Lustre: lustre-OST0000: new disk, initializing
[ 3281.660055] Lustre: srv-lustre-OST0000: No data found on store. Initialize space.
[ 3281.780267] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost
[ 3281.783359] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost]
[ 3281.794438] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401
[ 3282.866935] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3286.254204] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5
[ 3286.258018] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro
[ 3286.279733] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5
[ 3286.282809] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 3286.305153] Lustre: lustre-OST0001: new disk, initializing
[ 3286.306556] Lustre: srv-lustre-OST0001: No data found on store. Initialize space.
[ 3287.386384] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost
[ 3287.389456] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost]
[ 3287.400141] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401
[ 3287.633514] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3291.381399] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 3303.715164] Lustre: DEBUG MARKER: == sanity-quota test 50: Test if lfs find --projid works ========================================================== 06:39:17 (1744886357)
[ 3315.682130] Lustre: DEBUG MARKER: == sanity-quota test 51: Test project accounting with mv/cp ========================================================== 06:39:29 (1744886369)
[ 3331.116135] Lustre: DEBUG MARKER: == sanity-quota test 52: Rename normal file across project ID ========================================================== 06:39:45 (1744886385)
[ 3335.185165] Lustre: DEBUG MARKER: rename directory return 255
[ 3350.431417] Lustre: DEBUG MARKER: == sanity-quota test 53: Project inherit attribute could be cleared ========================================================== 06:40:04 (1744886404)
[ 3355.282706] Lustre: DEBUG MARKER: == sanity-quota test 54: basic lfs project interface test ========================================================== 06:40:09 (1744886409)
[ 3363.236995] Lustre: DEBUG MARKER: == sanity-quota test 55: Chgrp should be affected by group quota ========================================================== 06:40:17 (1744886417)
[ 3384.222257] LustreError: 15005:0:(qmt_entry.c:552:qmt_adjust_edquot()) $$$ set revoke_time explicitly  qmt:lustre-QMT0000 pool:dt-0x0 id:60001 enforced:1 hard:51200 soft:0 granted:51200 time:0 qunit: 1024 edquot:0 may_rel:0 revoke:0 default:no
[ 3395.558879] Lustre: DEBUG MARKER: == sanity-quota test 56: lfs quota -t should work well === 06:40:49 (1744886449)
[ 3401.721740] Lustre: DEBUG MARKER: == sanity-quota test 57: lfs project could tolerate errors ========================================================== 06:40:55 (1744886455)
[ 3416.060413] Lustre: DEBUG MARKER: == sanity-quota test 58: project ID should be kept for new mirrors created by FID ========================================================== 06:41:09 (1744886469)
[ 3451.294666] Lustre: DEBUG MARKER: == sanity-quota test 59: lfs project dosen't crash kernel with project disabled ========================================================== 06:41:45 (1744886505)
[ 3452.384205] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107
[ 3452.387090] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 3452.388742] Lustre: Skipped 11 previous similar messages
[ 3458.051483] Lustre: server umount lustre-MDT0000 complete
[ 3458.984440] LustreError: 18020:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744886514 with bad export cookie 16140171016237048490
[ 3458.985634] LustreError: MGC192.168.203.134@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail
[ 3458.990242] LustreError: 18020:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) Skipped 4 previous similar messages
[ 3459.127323] Lustre: server umount lustre-MDT0001 complete
[ 3469.686174] LustreError: 2145:0:(obd_class.h:479:obd_check_dev()) Device 19 not setup
[ 3469.688275] LustreError: 2145:0:(obd_class.h:479:obd_check_dev()) Skipped 67 previous similar messages
[ 3469.699227] Lustre: server umount lustre-OST0000 complete
[ 3480.712738] Lustre: server umount lustre-OST0001 complete
[ 3484.662302] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing load_modules_local
[ 3486.980840] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc
[ 3487.151056] LustreError: lustre-MDT0000: can't enable quota enforcement since space accounting isn't functional. Please run tunefs.lustre --quota on an unmounted filesystem if not done already
[ 3487.904425] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3490.017678] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 3490.096021] LustreError: lustre-MDT0001: can't enable quota enforcement since space accounting isn't functional. Please run tunefs.lustre --quota on an unmounted filesystem if not done already
[ 3490.826216] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3491.562591] Lustre: Modifying parameter general.debug_raw_pointers in log params
[ 3491.564590] Lustre: Skipped 3 previous similar messages
[ 3493.203574] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5
[ 3493.207486] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 3493.280736] LustreError: lustre-OST0000: can't enable quota enforcement since space accounting isn't functional. Please run tunefs.lustre --quota on an unmounted filesystem if not done already
[ 3494.450414] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3496.279386] LustreError: 5908:0:(ldlm_lib.c:1113:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server.
[ 3496.283688] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:18 to 0x280000401:33)
[ 3496.291242] LustreError: 5908:0:(ldlm_lib.c:1113:target_handle_connect()) Skipped 6 previous similar messages
[ 3496.614317] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5
[ 3496.617929] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 3496.654134] LustreError: lustre-OST0001: can't enable quota enforcement since space accounting isn't functional. Please run tunefs.lustre --quota on an unmounted filesystem if not done already
[ 3497.794524] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:22 to 0x2c0000401:65)
[ 3497.857665] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3504.015512] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 3511.109696] LustreError: 4093:0:(osd_handler.c:3264:osd_quota_transfer()) lustre-MDT0000: quota transfer failed. Is project enforcement enabled on the ldiskfs filesystem? rc = -95
[ 3514.816827] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping)
[ 3514.818717] Lustre: Skipped 11 previous similar messages
[ 3517.810348] Lustre: server umount lustre-MDT0000 complete
[ 3518.890420] LustreError: 4079:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1744886574 with bad export cookie 16140171016237096692
[ 3518.895095] LustreError: 4079:0:(ldlm_lockd.c:2549:ldlm_cancel_handler()) Skipped 2 previous similar messages
[ 3519.016929] Lustre: server umount lustre-MDT0001 complete
[ 3530.030510] Lustre: server umount lustre-OST0000 complete
[ 3541.218734] Lustre: server umount lustre-OST0001 complete
[ 3545.176762] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing load_modules_local
[ 3547.341758] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc
[ 3547.481091] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180
[ 3547.483767] Lustre: Skipped 7 previous similar messages
[ 3548.249302] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3550.376944] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 3551.227511] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3553.610750] LDISKFS-fs (dm-2): file extents enabled, maximum tree depth=5
[ 3553.614985] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 3554.903180] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3557.010059] LDISKFS-fs (dm-3): file extents enabled, maximum tree depth=5
[ 3557.014100] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,acl,no_mbcache,nodelalloc
[ 3558.050552] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:22 to 0x2c0000401:97)
[ 3558.064660] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:35 to 0x280000401:65)
[ 3558.251363] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all
[ 3564.418100] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 3580.846426] Lustre: DEBUG MARKER: == sanity-quota test 60: Test quota for root with setgid ========================================================== 06:43:54 (1744886634)
[ 3583.053731] LustreError: 15670:0:(qsd_reint.c:617:qqi_reint_delayed()) lustre-MDT0000: Delaying reintegration for qtype:0 until pending updates are flushed.
[ 3583.062442] LustreError: 15670:0:(qsd_reint.c:617:qqi_reint_delayed()) Skipped 7 previous similar messages
[ 3595.980146] Lustre: DEBUG MARKER: SKIP: sanity-quota test_61 skipping SLOW test 61
[ 3596.361622] Lustre: DEBUG MARKER: == sanity-quota test 62: Project inherit should be only changed by root ========================================================== 06:44:10 (1744886650)
[ 3601.073362] Lustre: DEBUG MARKER: SKIP: sanity-quota test_63 skipping excluded test 63
[ 3601.445126] Lustre: DEBUG MARKER: == sanity-quota test 64: lfs project on non dir/files should succeed ========================================================== 06:44:15 (1744886655)
[ 3613.554252] Lustre: DEBUG MARKER: SKIP: sanity-quota test_65 skipping excluded test 65
[ 3613.918616] Lustre: DEBUG MARKER: == sanity-quota test 66: nonroot user can not change project state in default ========================================================== 06:44:27 (1744886667)
[ 3626.553072] Lustre: DEBUG MARKER: == sanity-quota test 67: quota pools recalculation ======= 06:44:40 (1744886680)
[ 3629.043265] Lustre: DEBUG MARKER: User quota (block hardlimit:20 MB)
[ 3630.485293] Lustre: 21555:0:(qsd_reint.c:240:qsd_reint_index()) lustre-OST0001: index version for fid [0x200000005:0x100a:0x0] is 0, but index isn't empty (1)
[ 3630.490135] Lustre: 21555:0:(qsd_reint.c:240:qsd_reint_index()) Skipped 2 previous similar messages
[ 3631.558518] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475
[ 3633.036665] Lustre: DEBUG MARKER: oleg334-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475
[ 3634.101923] Lustre: DEBUG MARKER: Write...
[ 3641.071590] LustreError: 23039:0:(mgs_handler.c:848:mgs_iocontrol_pool()) OBD_IOC_POOL err -17, cmd CE021 for pool lustre.qpool1
[ 3644.364816] Lustre: DEBUG MARKER: Write...
[ 3646.625587] Lustre: DEBUG MARKER: Write...
[ 3679.418166] Lustre: DEBUG MARKER: == sanity-quota test 68: slave number in quota pool changed after each add/remove OST ========================================================== 06:45:33 (1744886733)
[ 3688.064678] LustreError: 28048:0:(mgs_handler.c:848:mgs_iocontrol_pool()) OBD_IOC_POOL err -17, cmd CE021 for pool lustre.qpool1
[ 3704.853530] Lustre: DEBUG MARKER: == sanity-quota test 69: EDQUOT at one of pools shouldn't affect DOM ========================================================== 06:45:58 (1744886758)
[ 3707.653785] LustreError: 30685:0:(qsd_reint.c:617:qqi_reint_delayed()) lustre-MDT0000: Delaying reintegration for qtype:2 until pending updates are flushed.
[ 3707.662097] LustreError: 30685:0:(qsd_reint.c:617:qqi_reint_delayed()) Skipped 7 previous similar messages
[ 3713.665236] Lustre: DEBUG MARKER: User quota (block hardlimit:200 MB)
[ 3714.012412] Lustre: DEBUG MARKER: User quota (block hardlimit:10 MB)
[ 3736.498664] Lustre: DEBUG MARKER: == sanity-quota test 70a: check lfs setquota/quota with a pool option ========================================================== 06:46:30 (1744886790)
[ 3752.313126] Lustre: DEBUG MARKER: == sanity-quota test 70b: lfs setquota pool works properly ========================================================== 06:46:46 (1744886806)
[ 3767.773516] Lustre: DEBUG MARKER: == sanity-quota test 71a: Check PFL with quota pools ===== 06:47:01 (1744886821)
[ 3770.287291] Lustre: DEBUG MARKER: User quota (block hardlimit:100 MB)
[ 3781.543430] Lustre: DEBUG MARKER: Write...
[ 3782.038644] Lustre: DEBUG MARKER: Files for user (quota_usr), count=1:
[ 3782.390066] Lustre: DEBUG MARKER: sanity-quota test_71a: @@@@@@ FAIL: user write failure, but expect success