-----============= acceptance-small: sanity-quota ============----- Tue Jun 23 09:54:21 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg140-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanity-quota.*ex': No such file or directory excepting tests: 2 4a 63 65 skipping tests SLOW=no: 61 12a 9 === sanity-quota: start setup 09:54:49 (1782222889) === oleg140-client.virtnet: executing check_config_client /mnt/lustre oleg140-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg140-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff973b8672c000.idle_timeout=debug osc.lustre-OST0001-osc-ffff973b8672c000.idle_timeout=debug oleg140-server: oleg140-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanity-quota: finish setup 09:55:30 (1782222930) === using SAVE_PROJECT_SUPPORTED=0 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [true] running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [touch] [/mnt/lustre/d0_runas_test/f7626] running as uid/gid/euid/egid 60001/60001/60001/60001, groups: 60001 [true] running as uid/gid/euid/egid 60001/60001/60001/60001, groups: 60001 [touch] [/mnt/lustre/d0_runas_test/f7626] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 -u quota_usr space:0 -u quota_2usr space:0 -g quota_usr space:0 -g quota_2usr space:0 -p 1000 space:0 -p 1001 space:0 -u quota_usr inodes:0 -u quota_2usr inodes:0 -g quota_usr inodes:0 -g quota_2usr inodes:0 -p 1000 inodes:0 -p 1001 inodes:0 == sanity-quota test 0: Test basic quota performance ===== 09:56:50 (1782223010) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d0.sanity-quota/f0.sanity-quota-0] [count=10] [conv=fsync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 2.66704 s, 3.9 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d0.sanity-quota/f0.sanity-quota-0] [count=10] [conv=fsync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 2.64571 s, 4.0 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 0 (73s) == sanity-quota test 1a: Block hard limit (normal use and out of quota) ========================================================== 09:58:03 (1782223083) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:10 MB) -u quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.13591 s, 4.6 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-0] [count=5] [seek=5] dd: error writing '/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-0': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 1.4824 s, 2.8 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.299268 s, 0.0 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u quota_usr space:0 -------------------------------------- Group quota (block hardlimit:10 MB) -g quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-1] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.58063 s, 3.3 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-1] [count=5] [seek=5] dd: error writing '/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-1': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 1.66467 s, 2.5 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-1] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-1': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.33291 s, 0.0 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -g quota_usr space:0 -p 1000 space:0 -------------------------------------- Project quota (block hardlimit:10 mb) /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-2 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-2] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.19686 s, 4.4 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-2] [count=5] [seek=5] dd: error writing '/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-2': Disk quota exceeded 5+0 records in 4+0 records out 4386816 bytes (4.4 MB, 4.2 MiB) copied, 2.02598 s, 2.2 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-2] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-2': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.368923 s, 0.0 kB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -p 1000 space:0 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1a (245s) == sanity-quota test 1b: Quota pools: Block hard limit (normal use and out of quota) ========================================================== 10:02:09 (1782223329) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:20 MB) Creating new pool qpool1 oleg140-server: Pool lustre.qpool1 created Adding targets to pool oleg140-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg140-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 -u quota_usr space:0 used 0 -u quota_usr bhardlimit:10240 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.33744 s, 3.9 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-0] [count=5] [seek=5] dd: error writing '/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-0': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 1.6161 s, 2.6 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.384943 s, 0.0 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u quota_usr space:0 debugfs 1.47.3-wc2 (11-Nov-2025) debugfs: No such file or directory while trying to open /tmp/lustre-mdt1 ls: Filesystem not open debugfs 1.47.3-wc2 (11-Nov-2025) debugfs: No such file or directory while trying to open /tmp/lustre-mdt1 ls: Filesystem not open -------------------------------------- Group quota (block hardlimit:20 MB) -g quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-1] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.40471 s, 3.7 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-1] [count=5] [seek=5] dd: error writing '/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-1': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 1.4509 s, 2.9 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-1] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-1': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.229001 s, 0.0 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -g quota_usr space:0 -p 1000 space:0 -------------------------------------- Project quota (block hardlimit:20 mb) /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-2 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-2] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.50058 s, 3.5 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-2] [count=5] [seek=5] dd: error writing '/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-2': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 1.06785 s, 3.9 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-2] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-2': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.306552 s, 0.0 kB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -p 1000 space:0 Destroy the created pools: qpool1 lustre.qpool1 oleg140-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg140-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg140-server: Pool lustre.qpool1 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1b (267s) == sanity-quota test 1c: Quota pools: check 3 pools with hardlimit only for global ========================================================== 10:06:36 (1782223596) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:20 MB) Creating new pool qpool1 oleg140-server: Pool lustre.qpool1 created Adding targets to pool oleg140-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg140-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 Creating new pool qpool2 oleg140-server: Pool lustre.qpool2 created Waiting 90s for '' Adding targets to pool oleg140-server: OST lustre-OST0000_UUID added to pool lustre.qpool2 oleg140-server: OST lustre-OST0001_UUID added to pool lustre.qpool2 -u quota_usr space:0 used 0 -u quota_usr bhardlimit:20480 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1c.sanity-quota/f1c.sanity-quota-0] [count=10] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 2.73274 s, 3.8 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1c.sanity-quota/f1c.sanity-quota-0] [count=10] [seek=10] dd: error writing '/mnt/lustre/d1c.sanity-quota/f1c.sanity-quota-0': Disk quota exceeded 10+0 records in 9+0 records out 9437184 bytes (9.4 MB, 9.0 MiB) copied, 2.928 s, 3.2 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1c.sanity-quota/f1c.sanity-quota-0] [count=1] [seek=20] dd: error writing '/mnt/lustre/d1c.sanity-quota/f1c.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.212624 s, 0.0 kB/s -u quota_usr space:19462 qpool1 used 19462 -u quota_usr space:19462 qpool2 used 19462 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u quota_usr space:0 Destroy the created pools: qpool1,qpool2 lustre.qpool1 oleg140-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg140-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg140-server: Pool lustre.qpool1 destroyed lustre.qpool2 oleg140-server: OST lustre-OST0000_UUID removed from pool lustre.qpool2 oleg140-server: OST lustre-OST0001_UUID removed from pool lustre.qpool2 oleg140-server: Pool lustre.qpool2 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1c (170s) == sanity-quota test 1d: Quota pools: check block hardlimit on different pools ========================================================== 10:09:27 (1782223767) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:20 MB) Creating new pool qpool1 oleg140-server: Pool lustre.qpool1 created Adding targets to pool oleg140-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg140-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 Creating new pool qpool2 oleg140-server: Pool lustre.qpool2 created Adding targets to pool oleg140-server: OST lustre-OST0000_UUID added to pool lustre.qpool2 oleg140-server: OST lustre-OST0001_UUID added to pool lustre.qpool2 -u quota_usr space:0 used 0 -u quota_usr bhardlimit:20480 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1d.sanity-quota/f1d.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.886193 s, 5.9 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1d.sanity-quota/f1d.sanity-quota-0] [count=5] [seek=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.890948 s, 5.9 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1d.sanity-quota/f1d.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1d.sanity-quota/f1d.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.319542 s, 0.0 kB/s -u quota_usr space:10244 qpool1 used 10244 -u quota_usr space:10244 qpool2 used 10244 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u quota_usr space:0 Destroy the created pools: qpool1,qpool2 lustre.qpool1 oleg140-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg140-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg140-server: Pool lustre.qpool1 destroyed lustre.qpool2 oleg140-server: OST lustre-OST0000_UUID removed from pool lustre.qpool2 oleg140-server: OST lustre-OST0001_UUID removed from pool lustre.qpool2 oleg140-server: Pool lustre.qpool2 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1d (166s) == sanity-quota test 1e: Quota pools: global pool high block limit vs quota pool with small ========================================================== 10:12:12 (1782223932) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:53000000 MB) Creating new pool qpool1 oleg140-server: Pool lustre.qpool1 created Adding targets to pool oleg140-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 -u quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1e.sanity-quota/f1e.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.52818 s, 3.4 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1e.sanity-quota/f1e.sanity-quota-0] [count=5] [seek=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.3981 s, 3.8 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1e.sanity-quota/f1e.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1e.sanity-quota/f1e.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.411564 s, 0.0 kB/s Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1e.sanity-quota/f1e.sanity-quota-1] [count=20] 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 3.86499 s, 5.4 MB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u quota_usr space:0 Destroy the created pools: qpool1 lustre.qpool1 oleg140-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg140-server: Pool lustre.qpool1 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1e (131s) == sanity-quota test 1f: Quota pools: correct qunit after removing/adding OST ========================================================== 10:14:23 (1782224063) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:200 MB) Creating new pool qpool1 oleg140-server: Pool lustre.qpool1 created Adding targets to pool oleg140-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 -u quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.07768 s, 4.9 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0] [count=5] [seek=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.41259 s, 3.7 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.217796 s, 0.0 kB/s Removing lustre-OST0000_UUID from qpool1 oleg140-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 pdsh@oleg140-client: oleg140-server: ssh exited with exit code 1 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Adding targets to pool oleg140-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.39567 s, 3.8 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0] [count=5] [seek=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.22556 s, 4.3 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.33261 s, 0.0 kB/s Destroy the created pools: qpool1 lustre.qpool1 oleg140-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg140-server: Pool lustre.qpool1 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 1f (157s) == sanity-quota test 1g: Quota pools: Block hard limit with wide striping ========================================================== 10:17:00 (1782224220) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 old_mdmb 462 mdmb 1 osc.lustre-OST0000-osc-ffff973b8672c000.max_dirty_mb=1 osc.lustre-OST0001-osc-ffff973b8672c000.max_dirty_mb=1 User quota (block hardlimit:40 MB) Creating new pool qpool1 oleg140-server: Pool lustre.qpool1 created Adding targets to pool oleg140-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg140-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 -u quota_usr space:0 used 0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1g.sanity-quota/f1g.sanity-quota-0] [count=10] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 7.78478 s, 1.3 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1g.sanity-quota/f1g.sanity-quota-0] [count=10] [seek=10] dd: error writing '/mnt/lustre/d1g.sanity-quota/f1g.sanity-quota-0': Disk quota exceeded 9+0 records in 8+0 records out 8470528 bytes (8.5 MB, 8.1 MiB) copied, 7.14745 s, 1.2 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1g.sanity-quota/f1g.sanity-quota-0] [count=2] [seek=20] [oflag=sync] dd: error writing '/mnt/lustre/d1g.sanity-quota/f1g.sanity-quota-0': Disk quota exceeded 2+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 1.48505 s, 706 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u quota_usr space:0 Destroy the created pools: qpool1 lustre.qpool1 oleg140-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg140-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg140-server: Pool lustre.qpool1 destroyed osc.lustre-OST0000-osc-ffff973b8672c000.max_dirty_mb=462 osc.lustre-OST0001-osc-ffff973b8672c000.max_dirty_mb=462 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1g (154s) == sanity-quota test 1h: Block hard limit test using fallocate ========================================================== 10:19:34 (1782224374) fallocate on zfs doesn't consume space fallocate not supported SKIP: sanity-quota test_1h need >= 2.13.57 and ldiskfs for fallocate SKIP 1h (4s) == sanity-quota test 1i: Quota pools: different limit and usage relations ========================================================== 10:19:38 (1782224378) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:200 MB) Creating new pool qpool1 oleg140-server: Pool lustre.qpool1 created Adding targets to pool oleg140-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 -u quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.62873 s, 3.2 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0] [count=5] [seek=5] dd: error writing '/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 1.1252 s, 3.7 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.29209 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 9220 0 0 - 1 0 0 - Pool: lustre.qpool1 lustre-OST0000 9220 - 10240 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 10240 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-1] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.26337 s, 4.1 MB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.54943 s, 3.4 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0] [count=5] [seek=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.69473 s, 3.1 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.25626 s, 0.0 kB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-1] [count=3] 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.595329 s, 5.3 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-2] [count=3] dd: error writing '/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-2': Disk quota exceeded 2+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.46055 s, 2.3 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-2] [seek=3] [count=1] dd: error writing '/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-2': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.269298 s, 0.0 kB/s Destroy the created pools: qpool1 lustre.qpool1 oleg140-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg140-server: Pool lustre.qpool1 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 1i (150s) == sanity-quota test 1j: Enable project quota enforcement for root ========================================================== 10:22:08 (1782224528) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -------------------------------------- Project quota (hardlimit: 40 mb, 2048 files) /home/green/git/lustre-release/lustre/utils/lfs project -sp 1000 /mnt/lustre/d1j.sanity-quota/test_dir /home/green/git/lustre-release/lustre/utils/lfs project -sp 1000 /mnt/lustre/d1j.sanity-quota/test_dir1 /home/green/git/lustre-release/lustre/utils/lfs project -sp 1001 /mnt/lustre/d1j.sanity-quota/test_dir2 /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-0 /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-1 /home/green/git/lustre-release/lustre/utils/lfs project -p 1001 /mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-2 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 40960 - 0 0 2048 - lustre-MDT0000 0 - 0 - 0 - 1024 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 1024, total allocated block limit: 0 Disk quotas for prj (pid 1001): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 81920 - 0 0 4096 - lustre-MDT0000 0 - 0 - 0 - 1024 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 1024, total allocated block limit: 0 running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir/tfile1-] [2048] - create 676 (time 1782224570.98 total 10.00 last 67.57) - create 1171 (time 1782224581.00 total 20.02 last 49.42) - create 1590 (time 1782224591.01 total 30.03 last 41.87) - create 2017 (time 1782224601.03 total 40.05 last 42.62) total: 2048 create in 40.88 seconds: 50.10 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir/tfile2-] [048] total: 4 create in 0.11 seconds: 37.78 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-0] [count=40] [oflag=direct] 40+0 records in 40+0 records out 41943040 bytes (42 MB, 40 MiB) copied, 7.52467 s, 5.6 MB/s running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-0] [count=20] [seek=40] [oflag=direct] 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 4.41456 s, 4.8 MB/s osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=1 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=1 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=1 error: get_param: param_path 'osd-*/*/quota_slave/root_prj_enable': No such file or directory Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 64768* 0 40960 - 2056* 0 2048 - lustre-MDT0000 3319 - 0 - 2056 - 2056 - lustre-OST0000 61449* - 61449 - 1* - 0 - lustre-OST0001 1 - 1025 - 1 - 0 - Total allocated inode limit: 2056, total allocated block limit: 62474 Disk quotas for prj (pid 1001): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 10 0 81920 - 2 0 4096 - lustre-MDT0000 9 - 0 - 2 - 1024 - lustre-OST0000 1 - 16384 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 1024, total allocated block limit: 16384 running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir1/tfile-] [2048] mknod(/mnt/lustre/d1j.sanity-quota/test_dir1/tfile-0) error: Disk quota exceeded total: 0 create in 0.06 seconds: 0.00 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-1] [count=40] [oflag=direct] dd: error writing '/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-1': Disk quota exceeded 2+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.299497 s, 3.5 MB/s running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-1] [count=20] [seek=40] [oflag=direct] dd: error writing '/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-1': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0834184 s, 0.0 kB/s running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir2/tfile1-] [2048] - create 670 (time 1782224629.57 total 10.00 last 66.97) - create 1265 (time 1782224639.60 total 20.03 last 59.35) - create 1933 (time 1782224649.61 total 30.04 last 66.74) total: 2048 create in 31.62 seconds: 64.77 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir2/tfile2-] [048] total: 4 create in 0.07 seconds: 60.30 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-2] [count=40] [oflag=direct] 40+0 records in 40+0 records out 41943040 bytes (42 MB, 40 MiB) copied, 7.25413 s, 5.8 MB/s running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-2] [count=20] [seek=40] [oflag=direct] 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 3.48098 s, 6.0 MB/s osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=0 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=0 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=0 On MGS 192.168.201.140, active = nodemap.active=1 On MGS 192.168.201.140, default.admin_nodemap = nodemap.default.admin_nodemap=1 On MGS 192.168.201.140, default.trusted_nodemap = nodemap.default.trusted_nodemap=1 On MGS 192.168.201.140, test_1j.admin_nodemap = nodemap.test_1j.admin_nodemap=1 On MGS 192.168.201.140, test_1j.trusted_nodemap = nodemap.test_1j.trusted_nodemap=1 On MGS 192.168.201.140, test_1j.rbac = nodemap.test_1j.rbac=file_perms,dne_ops,quota_ops,byfid_ops,chlg_ops running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir/tfile3-] [2048] mknod(/mnt/lustre/d1j.sanity-quota/test_dir/tfile3-0) error: Disk quota exceeded total: 0 create in 0.04 seconds: 0.00 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-0] [count=0] [seek=] [oflag=direct] dd: invalid number: '' On MGS 192.168.201.140, active = nodemap.active=0 running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir/tfile4-] [2048] - create 595 (time 1782224718.88 total 10.01 last 59.44) - create 1139 (time 1782224728.89 total 20.02 last 54.37) - create 1537 (time 1782224738.91 total 30.04 last 39.71) - create 1953 (time 1782224748.95 total 40.08 last 41.42) total: 2048 create in 41.89 seconds: 48.89 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-0] [count=40] [seek=40] [oflag=direct] 40+0 records in 40+0 records out 41943040 bytes (42 MB, 40 MiB) copied, 8.02435 s, 5.2 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -p 1000 space:0 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=0 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=0 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=0 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1j (487s) == sanity-quota test 1k: LQA: Block hard limit (normal use and out of quota) ========================================================== 10:30:15 (1782225015) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 range_min 500, range_max 60100 Disk default usr quota: Filesystem bquota blimit bgrace iquota ilimit igrace /mnt/lustre 0 20480 - 0 0 - Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=10] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 2.62535 s, 4.0 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=10] [seek=10] dd: error writing '/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0': Disk quota exceeded 10+0 records in 9+0 records out 9437184 bytes (9.4 MB, 9.0 MiB) copied, 2.66786 s, 3.5 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=1] [seek=20] dd: error writing '/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.314175 s, 0.0 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=10] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 2.67003 s, 3.9 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=10] [seek=10] dd: error writing '/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0': Disk quota exceeded 10+0 records in 9+0 records out 9437184 bytes (9.4 MB, 9.0 MiB) copied, 2.43208 s, 3.9 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=1] [seek=20] dd: error writing '/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.413441 s, 0.0 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=10] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 3.22573 s, 3.3 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=10] [seek=10] dd: error writing '/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0': Disk quota exceeded 10+0 records in 9+0 records out 9437184 bytes (9.4 MB, 9.0 MiB) copied, 2.40282 s, 3.9 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=1] [seek=20] dd: error writing '/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.234799 s, 0.0 kB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 1k (185s) == sanity-quota test 1l: Async writes should not be rejected by quota with root_prj_enable ========================================================== 10:33:20 (1782225200) Starting client: oleg140-client.virtnet: -o user_xattr,flock 192.168.201.140@tcp:/lustre /mnt/lustre2 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 /home/green/git/lustre-release/lustre/utils/lfs project -sp 1000 /mnt/lustre/d1l.sanity-quota osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=1 dd: error writing '/mnt/lustre/d1l.sanity-quota/f1l.sanity-quota.1': Disk quota exceeded 29+0 records in 28+0 records out 29360128 bytes (29 MB, 28 MiB) copied, 3.85916 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0903296 s, 11.6 MB/s osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete 192.168.201.140@tcp:/lustre /mnt/lustre2 lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg140-client.virtnet /mnt/lustre2 (opts:) PASS 1l (69s) SKIP: sanity-quota test_2 skipping excluded test 2 == sanity-quota test 3a: Block soft limit (start timer, timer goes off, stop timer) ========================================================== 10:34:32 (1782225272) old_mdmb 462 mdmb 1 osc.lustre-OST0000-osc-ffff973b8672c000.max_dirty_mb=1 osc.lustre-OST0001-osc-ffff973b8672c000.max_dirty_mb=1 User quota (soft limit:4 MB grace:60 seconds) -u quota_usr space:0 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.910614 s, 4.6 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0] [bs=1K] [count=10] [seek=4096] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 1.32409 s, 7.7 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5169* 4096 0 56s 2* 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 5124 - 6148 - 1 - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 6232 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5169 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 5124 - 0 - 1 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0] [bs=1K] [count=10] [seek=5120] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.932004 s, 11.0 kB/s Grace time is 55s Sleep through grace ... ...sleep 60 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6193* 4096 0 expired 2* 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6148 - 7172 - 1 - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 7256 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6193 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6148 - 0 - 1 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0] [bs=1K] [count=4096] [seek=6144] dd: error writing '/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0': Disk quota exceeded 2+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.109423 s, 9.4 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0] [bs=1K] [count=10] [seek=10240] dd: error writing '/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0240615 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7217* 4096 0 expired 2* 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7172* - 7172 - 1* - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 7256 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7217 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7172 - 0 - 1 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 44 4096 0 - 1 0 0 - lustre-MDT0000 1* - 1 - 1* - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 84 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 44 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.791566 s, 5.3 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Group quota (soft limit:4 MB grace:60 seconds) -g quota_usr space:0 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.643734 s, 6.5 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1] [bs=1K] [count=10] [seek=4096] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.807412 s, 12.7 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5131 0 0 - 1 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 5129 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5131* 4096 0 57s 1* 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 5129 - 6148 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6148 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1] [bs=1K] [count=10] [seek=5120] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.0104674 s, 978 kB/s Grace time is 57s Sleep through grace ... ...sleep 62 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215* 4096 0 expired 2* 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213* - 6213 - 2* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6213 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1] [bs=1K] [count=4096] [seek=6144] dd: error writing '/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0361428 s, 0.0 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1] [bs=1K] [count=10] [seek=10240] dd: error writing '/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0345668 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215* 4096 0 expired 2* 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213* - 6213 - 2* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6213 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 66 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 4096 0 - 1 0 0 - lustre-MDT0000 1* - 1 - 1* - 0 - lustre-OST0000 66 - 1090 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 1090 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.34015 s, 12.3 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Project quota (soft limit:4 MB grace:60 sec) -p 1000 space:0 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 1.28745 s, 3.3 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2] [bs=1K] [count=10] [seek=4096] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.68354 s, 15.0 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5131 0 0 - 1 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 5124 - 0 - 1 - 0 - lustre-OST0001 6 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5131 0 0 - 1 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 5124 - 0 - 1 - 0 - lustre-OST0001 6 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5125* 4096 0 57s 1* 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 5124 - 6148 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6148 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2] [bs=1K] [count=10] [seek=5120] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.694355 s, 14.7 kB/s Grace time is 56s Sleep through grace ... ...sleep 61 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6148 - 0 - 1 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6148 - 0 - 1 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6149* 4096 0 expired 1* 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 6148 - 7172 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 7172 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2] [bs=1K] [count=4096] [seek=6144] dd: error writing '/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2': Disk quota exceeded 2+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.113865 s, 9.0 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2] [bs=1K] [count=10] [seek=10240] dd: error writing '/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0383461 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7239 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7172 - 0 - 1 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7239 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7172 - 0 - 1 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7173* 4096 0 expired 1* 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 7172* - 7172 - 1* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 7172 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 4096 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2 Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.699954 s, 6.0 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete osc.lustre-OST0000-osc-ffff973b8672c000.max_dirty_mb=462 osc.lustre-OST0001-osc-ffff973b8672c000.max_dirty_mb=462 PASS 3a (529s) == sanity-quota test 3b: Quota pools: Block soft limit (start timer, expires, stop timer) ========================================================== 10:43:21 (1782225801) limit 4 glbl_limit 8 grace 60 glbl_grace 120 old_mdmb 462 mdmb 1 osc.lustre-OST0000-osc-ffff973b8672c000.max_dirty_mb=1 osc.lustre-OST0001-osc-ffff973b8672c000.max_dirty_mb=1 User quota in qpool1(soft limit:4 MB grace:60 seconds) -u quota_usr space:0 Creating new pool qpool1 oleg140-server: Pool lustre.qpool1 created Adding targets to pool oleg140-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg140-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.525489 s, 8.0 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0] [bs=1K] [count=10] [seek=4096] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.741779 s, 13.8 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5131 8192 0 - 1 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 5129 - 6148 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6148 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5131 0 0 - 1 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 5129 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0] [bs=1K] [count=10] [seek=5120] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.0132194 s, 775 kB/s Quota info for qpool1: Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5131* 4096 0 57s 1* 0 0 - Grace time is 57s Sleep through grace ... ...sleep 62 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 8192 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213* - 6213 - 2* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6213 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0] [bs=1K] [count=4096] [seek=6144] dd: error writing '/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0279899 s, 0.0 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0] [bs=1K] [count=10] [seek=10240] dd: error writing '/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0199769 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 8192 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213* - 6213 - 2* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6213 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 8192 0 - 1 0 0 - lustre-MDT0000 1* - 1 - 1* - 0 - lustre-OST0000 66 - 1090 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 1090 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 66 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.376617 s, 11.1 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Group quota in qpool1(soft limit:4 MB grace:60 seconds) -g quota_usr space:0 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.450514 s, 9.3 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1] [bs=1K] [count=10] [seek=4096] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.849913 s, 12.0 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5124 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5126 8192 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 5124 - 6148 - 1 - 0 - lustre-OST0001 0 - 41 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6189 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1] [bs=1K] [count=10] [seek=5120] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.69292 s, 14.8 kB/s Quota info for qpool1: Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6150* 4096 0 57s 2* 0 0 - Grace time is 57s Sleep through grace ... ...sleep 62 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6148 - 0 - 1 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 8192 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6148 - 7172 - 1 - 0 - lustre-OST0001 66* - 66 - 1* - 0 - Total allocated inode limit: 0, total allocated block limit: 7238 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1] [bs=1K] [count=4096] [seek=6144] dd: error writing '/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1': Disk quota exceeded 2+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.135355 s, 7.6 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1] [bs=1K] [count=10] [seek=10240] dd: error writing '/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0141399 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7239 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7172 - 0 - 1 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7239 8192 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7172* - 7172 - 1* - 0 - lustre-OST0001 66* - 66 - 1* - 0 - Total allocated inode limit: 0, total allocated block limit: 7238 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 8192 0 - 1 0 0 - lustre-MDT0000 1* - 1 - 1* - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 66* - 66 - 1* - 0 - Total allocated inode limit: 0, total allocated block limit: 66 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.684672 s, 6.1 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Project quota in qpool1(soft:4 MB grace:60 sec) -p 1000 space:0 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.361078 s, 11.6 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2] [bs=1K] [count=10] [seek=4096] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 1.10705 s, 9.2 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5125 0 0 - 1 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 5124 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5125 0 0 - 1 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 5124 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5125 8192 0 - 1 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 5124 - 6148 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6148 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2] [bs=1K] [count=10] [seek=5120] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 1.29968 s, 7.9 kB/s Quota info for qpool1: Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6149* 4096 0 56s 1* 0 0 - Grace time is 56s Sleep through grace ... ...sleep 61 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6149 8192 0 - 1 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 6148 - 7172 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 7172 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2] [bs=1K] [count=4096] [seek=6144] dd: error writing '/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2': Disk quota exceeded 2+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.195998 s, 5.2 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2] [bs=1K] [count=10] [seek=10240] dd: error writing '/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0370384 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7239 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7237 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7239 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7237 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7173 8192 0 - 1 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 7172* - 7172 - 1* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 7172 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 66 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 66 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 8192 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2 Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.37547 s, 11.2 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Destroy the created pools: qpool1 lustre.qpool1 oleg140-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg140-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg140-server: Pool lustre.qpool1 destroyed osc.lustre-OST0000-osc-ffff973b8672c000.max_dirty_mb=462 osc.lustre-OST0001-osc-ffff973b8672c000.max_dirty_mb=462 PASS 3b (540s) == sanity-quota test 3c: Quota pools: check block soft limit on different pools ========================================================== 10:52:21 (1782226341) limit 4 limit2 8 glbl_limit 12 grace1 70 grace2 60 glbl_grace 80 old_mdmb 462 mdmb 1 osc.lustre-OST0000-osc-ffff973b8672c000.max_dirty_mb=1 osc.lustre-OST0001-osc-ffff973b8672c000.max_dirty_mb=1 User quota in qpool2(soft:8 MB grace:60 seconds) -u quota_usr space:0 Creating new pool qpool1 oleg140-server: Pool lustre.qpool1 created Adding targets to pool oleg140-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg140-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 Creating new pool qpool2 oleg140-server: Pool lustre.qpool2 created Adding targets to pool oleg140-server: OST lustre-OST0000_UUID added to pool lustre.qpool2 oleg140-server: OST lustre-OST0001_UUID added to pool lustre.qpool2 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0] [count=8] 8+0 records in 8+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 1.27909 s, 6.6 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0] [bs=1K] [count=10] [seek=8192] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 1.54592 s, 6.6 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 9265 12288 0 - 1 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 9220 - 10244 - 1 - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 10328 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 9265 0 0 - 1 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 9220 - 0 - 1 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m20s; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0] [bs=1K] [count=10] [seek=9216] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 1.14045 s, 9.0 kB/s Quota info for qpool2: Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 10289* 8192 0 54s 2* 0 0 - Grace time is 54s Sleep through grace ... ...sleep 59 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 10289 12288 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 10244 - 11268 - 1 - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 11352 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 10289 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 10244 - 0 - 1 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m20s; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0] [bs=1K] [count=4096] [seek=10240] dd: error writing '/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0': Disk quota exceeded 2+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.156329 s, 6.6 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0] [bs=1K] [count=10] [seek=14336] dd: error writing '/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0393717 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 11314 12288 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 11269* - 11269 - 1* - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 11353 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 11314 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 11269 - 0 - 1 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m20s; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 44 12288 0 - 1 0 0 - lustre-MDT0000 1* - 1 - 1* - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 84 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 44 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m20s; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0] [count=8] 8+0 records in 8+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 1.58236 s, 5.3 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Destroy the created pools: qpool1,qpool2 lustre.qpool1 oleg140-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg140-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg140-server: Pool lustre.qpool1 destroyed lustre.qpool2 oleg140-server: OST lustre-OST0000_UUID removed from pool lustre.qpool2 oleg140-server: OST lustre-OST0001_UUID removed from pool lustre.qpool2 oleg140-server: Pool lustre.qpool2 destroyed osc.lustre-OST0000-osc-ffff973b8672c000.max_dirty_mb=462 osc.lustre-OST0001-osc-ffff973b8672c000.max_dirty_mb=462 PASS 3c (229s) SKIP: sanity-quota test_4a skipping excluded test 4a == sanity-quota test 4b: Grace time strings handling ===== 10:56:13 (1782226573) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Valid grace strings test Block grace time: 1w3d; Inode grace time: 16m40s Block grace time: 5s; Inode grace time: 1w2d3h4m5s Invalid grace strings test lfs: bad inode-grace: 5c setquota failed: Unknown error -4 Set filesystem quotas. usage: setquota [-t] {-u|-U|-g|-G|-p|-P ID} {-b|-B|-i|-I LIMIT} [--lqa LQA_NAME] [--pool POOL] MOUNT_POINT setquota {-u|-g|-p ID} {--default|--delete} MOUNT_POINT lfs: bad inode-grace: 18446744073709551615 setquota failed: Unknown error -4 Set filesystem quotas. usage: setquota [-t] {-u|-U|-g|-G|-p|-P ID} {-b|-B|-i|-I LIMIT} [--lqa LQA_NAME] [--pool POOL] MOUNT_POINT setquota {-u|-g|-p ID} {--default|--delete} MOUNT_POINT lfs: bad inode-grace: -1 setquota failed: Unknown error -4 Set filesystem quotas. usage: setquota [-t] {-u|-U|-g|-G|-p|-P ID} {-b|-B|-i|-I LIMIT} [--lqa LQA_NAME] [--pool POOL] MOUNT_POINT setquota {-u|-g|-p ID} {--default|--delete} MOUNT_POINT PASS 4b (22s) == sanity-quota test 5: Chown & chgrp successfully even out of block/file quota ========================================================== 10:56:35 (1782226595) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Set quota limit (0 10M 0 1024) for quota_usr.quota_usr lfs setquota: warning: inode hardlimit '1024' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details lfs setquota: warning: inode hardlimit '1024' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details lfs setquota: warning: inode hardlimit '1024' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details -u quota_usr inodes:0 -g quota_usr inodes:0 -u quota_usr space:0 -g quota_usr space:0 -p 1000 inodes:0 -p 1000 space:0 Create more than 1024 files and more than 10 MB ... - create 976 (time 1782226671.11 total 10.00 last 97.59) total: 1025 create in 10.63 seconds: 96.45 ops/second /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d5.sanity-quota/f5.sanity-quota-0_1 11+0 records in 11+0 records out 11534336 bytes (12 MB, 11 MiB) copied, 0.77687 s, 14.8 MB/s Chown files to quota_usr.quota_usr ... - unlinked 0 (time 1782226725 ; total 0 ; last 0) total: 1025 unlinks in 11 seconds: 93.181816 unlinks/second Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 5 (168s) == sanity-quota test 6: Test dropping acquire request on master ========================================================== 10:59:23 (1782226763) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u quota_usr space:0 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_usr] [count=1] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199359 s, 5.3 MB/s running as uid/gid/euid/egid 60001/60001/60001/60001, groups: 60001 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_2usr] [count=1] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0647882 s, 16.2 MB/s at_max=20 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_usr] [count=3] [seek=1] [oflag=sync] [conv=notrunc] dd: error writing '/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_usr': Disk quota exceeded 3+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.506287 s, 4.1 MB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_val=601 fail_loc=0x513 old quota_slave timeout 10 osd-zfs.lustre-OST0000.quota_slave.timeout=10 osd-zfs.lustre-OST0001.quota_slave.timeout=10 running as uid/gid/euid/egid 60001/60001/60001/60001, groups: 60001 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_2usr] [count=3] [seek=1] [oflag=sync] [conv=notrunc] 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.478515 s, 6.6 MB/s Sleep for 41 seconds ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_usr] [count=3] [seek=1] [oflag=sync] [conv=notrunc] at_max=600 fail_val=0 fail_loc=0 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 55.563 s, 56.6 kB/s osd-zfs.lustre-OST0000.quota_slave.timeout=10 osd-zfs.lustre-OST0001.quota_slave.timeout=10 at_max=600 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 6 (165s) == sanity-quota test 7a: Quota reintegration (global index) ========================================================== 11:02:08 (1782226928) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u quota_usr space:0 -u quota_usr bhardlimit:0 Stop ost1... Stopping /mnt/lustre-ost1 (opts:) on oleg140-server Enable quota & set quota limit for quota_usr Start ost1... Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg140-server: oleg140-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg140-client: oleg140-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg140-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg140-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg140-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg140-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg140-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg140-server: *.lustre-OST0001.recovery_status status: INACTIVE running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7a.sanity-quota/f7a.sanity-quota] [count=6] [oflag=sync] dd: error writing '/mnt/lustre/d7a.sanity-quota/f7a.sanity-quota': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 2.3908 s, 1.8 MB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Stop ost1... Stopping /mnt/lustre-ost1 (opts:) on oleg140-server Start ost1... Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg140-server: oleg140-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg140-client: oleg140-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg140-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg140-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg140-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg140-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg140-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg140-server: *.lustre-OST0001.recovery_status status: INACTIVE running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7a.sanity-quota/f7a.sanity-quota] [count=6] [oflag=sync] 6+0 records in 6+0 records out 6291456 bytes (6.3 MB, 6.0 MiB) copied, 3.91456 s, 1.6 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 7a (298s) == sanity-quota test 7b: Quota reintegration (slave index) ========================================================== 11:07:06 (1782227226) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u quota_usr space:0 -u quota_usr bhardlimit:0 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7b.sanity-quota/f7b.sanity-quota] [count=1] [oflag=sync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.550945 s, 1.9 MB/s fail_val=0 fail_loc=0xa02 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7b.sanity-quota/f7b.sanity-quota] [count=1] [seek=1] [oflag=sync] [conv=notrunc] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.609226 s, 1.7 MB/s -u quota_usr bhardlimit:1026 fail_val=0 fail_loc=0 Restart ost to trigger reintegration... Stopping /mnt/lustre-ost1 (opts:) on oleg140-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg140-server: oleg140-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg140-client: oleg140-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg140-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg140-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg140-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg140-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg140-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg140-server: *.lustre-OST0001.recovery_status status: INACTIVE -u quota_usr bhardlimit:16777216 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 7b (231s) == sanity-quota test 7c: Quota reintegration (restart mds during reintegration) ========================================================== 11:10:57 (1782227457) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u quota_usr space:0 fail_val=0 fail_loc=0xa03 osd-zfs.lustre-OST0000.quota_slave.force_reint=1 osd-zfs.lustre-OST0001.quota_slave.force_reint=1 Stop mds... Stopping /mnt/lustre-mds1 (opts:) on oleg140-server fail_val=0 fail_loc=0 Start mds... Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg140-server: oleg140-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg140-client: oleg140-server: ssh exited with exit code 1 Started lustre-MDT0000 affected facets: ost1 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg140-server: *.lustre-OST0000.recovery_status status: COMPLETE Waiting 200s for 'glb[1],slv[1],reint[0]' Waiting 190s for 'glb[1],slv[1],reint[0]' Waiting 180s for 'glb[1],slv[1],reint[0]' Waiting 170s for 'glb[1],slv[1],reint[0]' Waiting 160s for 'glb[1],slv[1],reint[0]' Waiting 150s for 'glb[1],slv[1],reint[0]' Updated after 71s: want 'glb[1],slv[1],reint[0]' got 'glb[1],slv[1],reint[0]' affected facets: ost2 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg140-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg140-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg140-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg140-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg140-server: *.lustre-OST0001.recovery_status status: INACTIVE running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7c.sanity-quota/f7c.sanity-quota] [count=6] [oflag=sync] dd: error writing '/mnt/lustre/d7c.sanity-quota/f7c.sanity-quota': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 2.16702 s, 1.9 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 7c (238s) == sanity-quota test 7d: Quota reintegration (Transfer index in multiple bulks) ========================================================== 11:14:55 (1782227695) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 fail_val=0 fail_loc=0x608 affected facets: ost1 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg140-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg140-server: oleg140-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg140-server: *.lustre-OST0001.recovery_status status: INACTIVE fail_val=0 fail_loc=0 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7d.sanity-quota/f7d.sanity-quota] [count=21] [oflag=sync] dd: error writing '/mnt/lustre/d7d.sanity-quota/f7d.sanity-quota': Disk quota exceeded 20+0 records in 19+0 records out 19922944 bytes (20 MB, 19 MiB) copied, 7.79142 s, 2.6 MB/s running as uid/gid/euid/egid 60001/60001/60001/60001, groups: 60001 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7d.sanity-quota/f7d.sanity-quota-1] [count=21] [oflag=sync] dd: error writing '/mnt/lustre/d7d.sanity-quota/f7d.sanity-quota-1': Disk quota exceeded 20+0 records in 19+0 records out 19922944 bytes (20 MB, 19 MiB) copied, 7.2561 s, 2.7 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 7d (111s) == sanity-quota test 7e: Quota reintegration (inode limits) ========================================================== 11:16:46 (1782227806) SKIP: sanity-quota test_7e needs >= 2 MDTs SKIP 7e (5s)