-----============= acceptance-small: sanity-quota ============----- Wed Apr 22 10:54:10 EDT 2026 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg154-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanity-quota.*ex': No such file or directory excepting tests: 2 4a 63 65 skipping tests SLOW=no: 61 12a 9 === sanity-quota: start setup 10:54:17 (1776869657) === oleg154-client.virtnet: executing check_config_client /mnt/lustre oleg154-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg154-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8cbe04bfb800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8cbe04bfb800.idle_timeout=debug oleg154-server: oleg154-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanity-quota: finish setup 10:54:27 (1776869667) === using SAVE_PROJECT_SUPPORTED=0 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [true] running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [touch] [/mnt/lustre/d0_runas_test/f7528] running as uid/gid/euid/egid 60001/60001/60001/60001, groups: 60001 [true] running as uid/gid/euid/egid 60001/60001/60001/60001, groups: 60001 [touch] [/mnt/lustre/d0_runas_test/f7528] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 -u quota_usr space:0 -u quota_2usr space:0 -g quota_usr space:0 -g quota_2usr space:0 -p 1000 space:0 -p 1001 space:0 -u quota_usr inodes:0 -u quota_2usr inodes:0 -g quota_usr inodes:0 -g quota_2usr inodes:0 -p 1000 inodes:0 -p 1001 inodes:0 == sanity-quota test 0: Test basic quota performance ===== 10:54:47 (1776869687) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d0.sanity-quota/f0.sanity-quota-0] [count=10] [conv=fsync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.595725 s, 17.6 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d0.sanity-quota/f0.sanity-quota-0] [count=10] [conv=fsync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.626837 s, 16.7 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 0 (34s) == sanity-quota test 1a: Block hard limit (normal use and out of quota) ========================================================== 10:55:21 (1776869721) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:10 MB) -u quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.274686 s, 19.1 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-0] [count=5] [seek=5] dd: error writing '/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-0': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.294737 s, 14.2 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0715058 s, 0.0 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u quota_usr space:0 -------------------------------------- Group quota (block hardlimit:10 MB) -g quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-1] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.289305 s, 18.1 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-1] [count=5] [seek=5] dd: error writing '/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-1': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.311563 s, 13.5 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-1] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-1': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0676919 s, 0.0 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -g quota_usr space:0 -p 1000 space:0 -------------------------------------- Project quota (block hardlimit:10 mb) /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-2 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-2] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.295663 s, 17.7 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-2] [count=5] [seek=5] dd: error writing '/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-2': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.321541 s, 13.0 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-2] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-2': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0883884 s, 0.0 kB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -p 1000 space:0 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1a (120s) == sanity-quota test 1b: Quota pools: Block hard limit (normal use and out of quota) ========================================================== 10:57:21 (1776869841) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:20 MB) Creating new pool qpool1 oleg154-server: Pool lustre.qpool1 created Adding targets to pool oleg154-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg154-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 Waiting 90s for 'lustre-OST0000_UUID lustre-OST0001_UUID ' -u quota_usr space:0 used 0 -u quota_usr bhardlimit:10240 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.280921 s, 18.7 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-0] [count=5] [seek=5] dd: error writing '/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-0': Disk quota exceeded 5+0 records in 4+0 records out 4583424 bytes (4.6 MB, 4.4 MiB) copied, 0.357235 s, 12.8 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0709328 s, 0.0 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u quota_usr space:0 debugfs 1.47.3-wc2 (11-Nov-2025) debugfs: No such file or directory while trying to open /tmp/lustre-mdt1 ls: Filesystem not open debugfs 1.47.3-wc2 (11-Nov-2025) debugfs: No such file or directory while trying to open /tmp/lustre-mdt1 ls: Filesystem not open -------------------------------------- Group quota (block hardlimit:20 MB) -g quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-1] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.328904 s, 15.9 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-1] [count=5] [seek=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.277694 s, 18.9 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-1] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-1': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0571142 s, 0.0 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -g quota_usr space:0 -p 1000 space:0 -------------------------------------- Project quota (block hardlimit:20 mb) /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-2 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-2] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.280834 s, 18.7 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-2] [count=5] [seek=5] dd: error writing '/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-2': Disk quota exceeded 5+0 records in 4+0 records out 4898816 bytes (4.9 MB, 4.7 MiB) copied, 0.325042 s, 15.1 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-2] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-2': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0673994 s, 0.0 kB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -p 1000 space:0 Destroy the created pools: qpool1 lustre.qpool1 oleg154-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg154-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg154-server: Pool lustre.qpool1 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1b (132s) == sanity-quota test 1c: Quota pools: check 3 pools with hardlimit only for global ========================================================== 10:59:34 (1776869974) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:20 MB) Creating new pool qpool1 oleg154-server: Pool lustre.qpool1 created Adding targets to pool oleg154-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg154-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 Creating new pool qpool2 oleg154-server: Pool lustre.qpool2 created Adding targets to pool oleg154-server: OST lustre-OST0000_UUID added to pool lustre.qpool2 oleg154-server: OST lustre-OST0001_UUID added to pool lustre.qpool2 Waiting 90s for 'lustre-OST0000_UUID lustre-OST0001_UUID ' -u quota_usr space:0 used 0 -u quota_usr bhardlimit:20480 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1c.sanity-quota/f1c.sanity-quota-0] [count=10] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.64477 s, 16.3 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1c.sanity-quota/f1c.sanity-quota-0] [count=10] [seek=10] dd: error writing '/mnt/lustre/d1c.sanity-quota/f1c.sanity-quota-0': Disk quota exceeded 10+0 records in 9+0 records out 9437184 bytes (9.4 MB, 9.0 MiB) copied, 0.661756 s, 14.3 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1c.sanity-quota/f1c.sanity-quota-0] [count=1] [seek=20] dd: error writing '/mnt/lustre/d1c.sanity-quota/f1c.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0717633 s, 0.0 kB/s -u quota_usr space:19462 qpool1 used 19462 -u quota_usr space:19462 qpool2 used 19462 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u quota_usr space:0 Destroy the created pools: qpool1,qpool2 lustre.qpool1 oleg154-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg154-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg154-server: Pool lustre.qpool1 destroyed lustre.qpool2 oleg154-server: OST lustre-OST0000_UUID removed from pool lustre.qpool2 oleg154-server: OST lustre-OST0001_UUID removed from pool lustre.qpool2 oleg154-server: Pool lustre.qpool2 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1c (77s) == sanity-quota test 1d: Quota pools: check block hardlimit on different pools ========================================================== 11:00:51 (1776870051) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:20 MB) Creating new pool qpool1 oleg154-server: Pool lustre.qpool1 created Waiting 90s for '' Adding targets to pool oleg154-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg154-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 Creating new pool qpool2 oleg154-server: Pool lustre.qpool2 created Waiting 90s for '' Adding targets to pool oleg154-server: OST lustre-OST0000_UUID added to pool lustre.qpool2 oleg154-server: OST lustre-OST0001_UUID added to pool lustre.qpool2 -u quota_usr space:0 used 0 -u quota_usr bhardlimit:20480 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1d.sanity-quota/f1d.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.274509 s, 19.1 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1d.sanity-quota/f1d.sanity-quota-0] [count=5] [seek=5] dd: error writing '/mnt/lustre/d1d.sanity-quota/f1d.sanity-quota-0': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.280419 s, 15.0 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1d.sanity-quota/f1d.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1d.sanity-quota/f1d.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.074389 s, 0.0 kB/s -u quota_usr space:9220 qpool1 used 9220 -u quota_usr space:9220 qpool2 used 9220 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u quota_usr space:0 Destroy the created pools: qpool1,qpool2 lustre.qpool1 oleg154-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg154-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg154-server: Pool lustre.qpool1 destroyed Waiting 90s for 'foo' lustre.qpool2 oleg154-server: OST lustre-OST0000_UUID removed from pool lustre.qpool2 oleg154-server: OST lustre-OST0001_UUID removed from pool lustre.qpool2 oleg154-server: Pool lustre.qpool2 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1d (78s) == sanity-quota test 1e: Quota pools: global pool high block limit vs quota pool with small ========================================================== 11:02:09 (1776870129) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:53000000 MB) Creating new pool qpool1 oleg154-server: Pool lustre.qpool1 created Adding targets to pool oleg154-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 -u quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1e.sanity-quota/f1e.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.369612 s, 14.2 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1e.sanity-quota/f1e.sanity-quota-0] [count=5] [seek=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.321096 s, 16.3 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1e.sanity-quota/f1e.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1e.sanity-quota/f1e.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0826452 s, 0.0 kB/s Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1e.sanity-quota/f1e.sanity-quota-1] [count=20] 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 1.13295 s, 18.5 MB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u quota_usr space:0 Destroy the created pools: qpool1 lustre.qpool1 oleg154-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg154-server: Pool lustre.qpool1 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1e (62s) == sanity-quota test 1f: Quota pools: correct qunit after removing/adding OST ========================================================== 11:03:11 (1776870191) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:200 MB) Creating new pool qpool1 oleg154-server: Pool lustre.qpool1 created Adding targets to pool oleg154-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 Waiting 90s for 'lustre-OST0000_UUID ' -u quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.296749 s, 17.7 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0] [count=5] [seek=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.296209 s, 17.7 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0663962 s, 0.0 kB/s Removing lustre-OST0000_UUID from qpool1 oleg154-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Adding targets to pool oleg154-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.28862 s, 18.2 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0] [count=5] [seek=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.269439 s, 19.5 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0532206 s, 0.0 kB/s Destroy the created pools: qpool1 lustre.qpool1 oleg154-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg154-server: Pool lustre.qpool1 destroyed Waiting 90s for 'foo' Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 1f (82s) == sanity-quota test 1g: Quota pools: Block hard limit with wide striping ========================================================== 11:04:33 (1776870273) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 old_mdmb 462 mdmb 1 osc.lustre-OST0000-osc-ffff8cbe04bfb800.max_dirty_mb=1 osc.lustre-OST0001-osc-ffff8cbe04bfb800.max_dirty_mb=1 User quota (block hardlimit:40 MB) Creating new pool qpool1 oleg154-server: Pool lustre.qpool1 created Adding targets to pool oleg154-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg154-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 -u quota_usr space:0 used 0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1g.sanity-quota/f1g.sanity-quota-0] [count=10] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 1.70742 s, 6.1 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1g.sanity-quota/f1g.sanity-quota-0] [count=10] [seek=10] dd: error writing '/mnt/lustre/d1g.sanity-quota/f1g.sanity-quota-0': Disk quota exceeded 9+0 records in 8+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 1.69129 s, 5.0 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1g.sanity-quota/f1g.sanity-quota-0] [count=2] [seek=20] [oflag=sync] dd: error writing '/mnt/lustre/d1g.sanity-quota/f1g.sanity-quota-0': Disk quota exceeded 2+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.31452 s, 3.3 MB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u quota_usr space:0 Destroy the created pools: qpool1 lustre.qpool1 oleg154-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg154-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg154-server: Pool lustre.qpool1 destroyed osc.lustre-OST0000-osc-ffff8cbe04bfb800.max_dirty_mb=462 osc.lustre-OST0001-osc-ffff8cbe04bfb800.max_dirty_mb=462 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1g (61s) == sanity-quota test 1h: Block hard limit test using fallocate ========================================================== 11:05:34 (1776870334) fallocate on zfs doesn't consume space fallocate not supported SKIP: sanity-quota test_1h need >= 2.13.57 and ldiskfs for fallocate SKIP 1h (1s) == sanity-quota test 1i: Quota pools: different limit and usage relations ========================================================== 11:05:35 (1776870335) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:200 MB) Creating new pool qpool1 oleg154-server: Pool lustre.qpool1 created Adding targets to pool oleg154-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 -u quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.337639 s, 15.5 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0] [count=5] [seek=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.278388 s, 18.8 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.064927 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 10244 0 0 - 1 0 0 - Pool: lustre.qpool1 lustre-OST0000 10244* - 10244 - 1* - 0 - Total allocated inode limit: 0, total allocated block limit: 10244 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-1] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.270765 s, 19.4 MB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.291153 s, 18.0 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0] [count=5] [seek=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.246455 s, 21.3 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0572355 s, 0.0 kB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-1] [count=3] 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.165215 s, 19.0 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-2] [count=3] dd: error writing '/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-2': Disk quota exceeded 2+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118125 s, 8.9 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-2] [seek=3] [count=1] dd: error writing '/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-2': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0593556 s, 0.0 kB/s Destroy the created pools: qpool1 lustre.qpool1 oleg154-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg154-server: Pool lustre.qpool1 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 1i (76s) == sanity-quota test 1j: Enable project quota enforcement for root ========================================================== 11:06:51 (1776870411) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Waiting 90s for 'ugp' Updated after 2s: want 'ugp' got 'ugp' -------------------------------------- Project quota (hardlimit: 40 mb, 2048 files) /home/green/git/lustre-release/lustre/utils/lfs project -sp 1000 /mnt/lustre/d1j.sanity-quota/test_dir /home/green/git/lustre-release/lustre/utils/lfs project -sp 1000 /mnt/lustre/d1j.sanity-quota/test_dir1 /home/green/git/lustre-release/lustre/utils/lfs project -sp 1001 /mnt/lustre/d1j.sanity-quota/test_dir2 /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-0 /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-1 /home/green/git/lustre-release/lustre/utils/lfs project -p 1001 /mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-2 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 40960 - 0 0 2048 - lustre-MDT0000 0 - 0 - 0 - 1024 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 1024, total allocated block limit: 0 Disk quotas for prj (pid 1001): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 81920 - 0 0 4096 - lustre-MDT0000 0 - 0 - 0 - 1024 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 1024, total allocated block limit: 0 running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir/tfile1-] [2048] total: 2048 create in 4.84 seconds: 423.47 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir/tfile2-] [048] total: 4 create in 0.01 seconds: 433.51 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-0] [count=40] [oflag=direct] 40+0 records in 40+0 records out 41943040 bytes (42 MB, 40 MiB) copied, 1.15767 s, 36.2 MB/s running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-0] [count=20] [seek=40] [oflag=direct] 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 0.586288 s, 35.8 MB/s osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=1 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=1 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=1 error: get_param: param_path 'osd-*/*/quota_slave/root_prj_enable': No such file or directory Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 63107* 0 40960 - 1024* 0 2048 - lustre-MDT0000 1658 - 0 - 1024 - 2048 - lustre-OST0000 61449* - 61449 - 1* - 0 - lustre-OST0001 1 - 1025 - 1 - 0 - Total allocated inode limit: 2048, total allocated block limit: 62474 Disk quotas for prj (pid 1001): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 10 0 81920 - 2 0 4096 - lustre-MDT0000 9 - 0 - 2 - 1024 - lustre-OST0000 1 - 16384 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 1024, total allocated block limit: 16384 running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir1/tfile-] [2048] mknod(/mnt/lustre/d1j.sanity-quota/test_dir1/tfile-0) error: Disk quota exceeded total: 0 create in 0.11 seconds: 0.00 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-1] [count=40] [oflag=direct] dd: error writing '/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-1': Disk quota exceeded 2+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0585456 s, 17.9 MB/s running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-1] [count=20] [seek=40] [oflag=direct] dd: error writing '/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-1': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0270145 s, 0.0 kB/s running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir2/tfile1-] [2048] total: 2048 create in 4.94 seconds: 414.94 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir2/tfile2-] [048] total: 4 create in 0.01 seconds: 442.57 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-2] [count=40] [oflag=direct] 40+0 records in 40+0 records out 41943040 bytes (42 MB, 40 MiB) copied, 1.19356 s, 35.1 MB/s running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-2] [count=20] [seek=40] [oflag=direct] 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 0.58156 s, 36.1 MB/s osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=0 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=0 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=0 On MGS 192.168.201.154, active = nodemap.active=1 On MGS 192.168.201.154, default.admin_nodemap = nodemap.default.admin_nodemap=1 On MGS 192.168.201.154, default.trusted_nodemap = nodemap.default.trusted_nodemap=1 On MGS 192.168.201.154, test_1j.admin_nodemap = nodemap.test_1j.admin_nodemap=1 On MGS 192.168.201.154, test_1j.trusted_nodemap = nodemap.test_1j.trusted_nodemap=1 On MGS 192.168.201.154, test_1j.rbac = nodemap.test_1j.rbac=file_perms,dne_ops,quota_ops,byfid_ops,chlg_ops running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir/tfile3-] [2048] mknod(/mnt/lustre/d1j.sanity-quota/test_dir/tfile3-0) error: Disk quota exceeded total: 0 create in 0.01 seconds: 0.00 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-0] [count=0] [seek=] [oflag=direct] dd: invalid number: '' On MGS 192.168.201.154, active = nodemap.active=0 running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir/tfile4-] [2048] total: 2048 create in 7.57 seconds: 270.40 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-0] [count=40] [seek=40] [oflag=direct] 40+0 records in 40+0 records out 41943040 bytes (42 MB, 40 MiB) copied, 1.38276 s, 30.3 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -p 1000 space:0 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=0 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=0 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=0 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1j (110s) == sanity-quota test 1k: LQA: Block hard limit (normal use and out of quota) ========================================================== 11:08:41 (1776870521) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 range_min 500, range_max 60100 Disk default usr quota: Filesystem bquota blimit bgrace iquota ilimit igrace /mnt/lustre 0 20480 - 0 0 - Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=10] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.583953 s, 18.0 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=10] [seek=10] dd: error writing '/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0': Disk quota exceeded 10+0 records in 9+0 records out 9437184 bytes (9.4 MB, 9.0 MiB) copied, 0.58363 s, 16.2 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=1] [seek=20] dd: error writing '/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0656778 s, 0.0 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=10] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.554154 s, 18.9 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=10] [seek=10] dd: error writing '/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0': Disk quota exceeded 10+0 records in 9+0 records out 9437184 bytes (9.4 MB, 9.0 MiB) copied, 0.525783 s, 17.9 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=1] [seek=20] dd: error writing '/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0564317 s, 0.0 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=10] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.5765 s, 18.2 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=10] [seek=10] dd: error writing '/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0': Disk quota exceeded 10+0 records in 9+0 records out 9437184 bytes (9.4 MB, 9.0 MiB) copied, 0.590784 s, 16.0 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=1] [seek=20] dd: error writing '/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0838951 s, 0.0 kB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 1k (100s) == sanity-quota test 1l: Async writes should not be rejected by quota with root_prj_enable ========================================================== 11:10:21 (1776870621) Starting client: oleg154-client.virtnet: -o user_xattr,flock 192.168.201.154@tcp:/lustre /mnt/lustre2 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 /home/green/git/lustre-release/lustre/utils/lfs project -sp 1000 /mnt/lustre/d1l.sanity-quota osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=1 dd: error writing '/mnt/lustre/d1l.sanity-quota/f1l.sanity-quota.1': Disk quota exceeded 30+0 records in 29+0 records out 30408704 bytes (30 MB, 29 MiB) copied, 0.680685 s, 44.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225306 s, 46.5 MB/s osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete 192.168.201.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg154-client.virtnet /mnt/lustre2 (opts:) PASS 1l (30s) SKIP: sanity-quota test_2 skipping excluded test 2 == sanity-quota test 3a: Block soft limit (start timer, timer goes off, stop timer) ========================================================== 11:10:51 (1776870651) old_mdmb 462 mdmb 1 osc.lustre-OST0000-osc-ffff8cbe04bfb800.max_dirty_mb=1 osc.lustre-OST0001-osc-ffff8cbe04bfb800.max_dirty_mb=1 User quota (soft limit:4 MB grace:60 seconds) -u quota_usr space:0 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.106362 s, 39.4 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0] [bs=1K] [count=10] [seek=4096] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.0955313 s, 107 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5167* 4096 0 59s 0* 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 6148 - 1 - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 6232 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5167 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 0 - 1 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0] [bs=1K] [count=10] [seek=5120] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.092835 s, 110 kB/s Grace time is 59s Sleep through grace ... ...sleep 64 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6193* 4096 0 expired 2* 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6148 - 7172 - 1 - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 7256 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6193 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6148 - 0 - 1 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0] [bs=1K] [count=4096] [seek=6144] dd: error writing '/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0': Disk quota exceeded 2+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0208837 s, 49.0 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0] [bs=1K] [count=10] [seek=10240] dd: error writing '/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.00400501 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7217* 4096 0 expired 2* 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7172* - 7172 - 1* - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 7256 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7217 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7172 - 0 - 1 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 44 4096 0 - 1 0 0 - lustre-MDT0000 1* - 1 - 1* - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 84 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 44 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.137202 s, 30.6 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Group quota (soft limit:4 MB grace:60 seconds) -g quota_usr space:0 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.136464 s, 30.7 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1] [bs=1K] [count=10] [seek=4096] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.124561 s, 82.2 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5129 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5129 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5129* 4096 0 59s 0* 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5129 - 6148 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6148 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1] [bs=1K] [count=10] [seek=5120] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.00473467 s, 2.2 MB/s Grace time is 59s Sleep through grace ... ...sleep 64 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215* 4096 0 expired 2* 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213* - 6213 - 2* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6213 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1] [bs=1K] [count=4096] [seek=6144] dd: error writing '/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.00543992 s, 0.0 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1] [bs=1K] [count=10] [seek=10240] dd: error writing '/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.00435673 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215* 4096 0 expired 2* 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213* - 6213 - 2* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6213 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 66 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 4096 0 - 1 0 0 - lustre-MDT0000 1* - 1 - 1* - 0 - lustre-OST0000 66 - 1090 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 1090 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.108437 s, 38.7 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Project quota (soft limit:4 MB grace:60 sec) -p 1000 space:0 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.110227 s, 38.1 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2] [bs=1K] [count=10] [seek=4096] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.109422 s, 93.6 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5129 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 0 - 1 - 0 - lustre-OST0001 6 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5129 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 0 - 1 - 0 - lustre-OST0001 6 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5124* 4096 0 59s 0* 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 6148 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6148 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2] [bs=1K] [count=10] [seek=5120] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.107571 s, 95.2 kB/s Grace time is 59s Sleep through grace ... ...sleep 64 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6148 - 0 - 1 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6148 - 0 - 1 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6149* 4096 0 expired 1* 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 6148 - 7172 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 7172 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2] [bs=1K] [count=4096] [seek=6144] dd: error writing '/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2': Disk quota exceeded 2+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0204693 s, 50.0 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2] [bs=1K] [count=10] [seek=10240] dd: error writing '/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.00389015 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7239 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7172 - 0 - 1 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7239 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7172 - 0 - 1 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7173* 4096 0 expired 1* 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 7172* - 7172 - 1* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 7172 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 4096 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2 Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.119242 s, 35.2 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete osc.lustre-OST0000-osc-ffff8cbe04bfb800.max_dirty_mb=462 osc.lustre-OST0001-osc-ffff8cbe04bfb800.max_dirty_mb=462 PASS 3a (371s) == sanity-quota test 3b: Quota pools: Block soft limit (start timer, expires, stop timer) ========================================================== 11:17:02 (1776871022) limit 4 glbl_limit 8 grace 60 glbl_grace 120 old_mdmb 462 mdmb 1 osc.lustre-OST0000-osc-ffff8cbe04bfb800.max_dirty_mb=1 osc.lustre-OST0001-osc-ffff8cbe04bfb800.max_dirty_mb=1 User quota in qpool1(soft limit:4 MB grace:60 seconds) -u quota_usr space:0 Creating new pool qpool1 oleg154-server: Pool lustre.qpool1 created Adding targets to pool oleg154-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg154-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.0996039 s, 42.1 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0] [bs=1K] [count=10] [seek=4096] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.1266 s, 80.9 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5129 8192 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5129 - 6148 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6148 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5129 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5129 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0] [bs=1K] [count=10] [seek=5120] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.00247068 s, 4.1 MB/s Quota info for qpool1: Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5129* 4096 0 59s 0* 0 0 - Grace time is 59s Sleep through grace ... ...sleep 64 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 8192 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213* - 6213 - 2* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6213 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0] [bs=1K] [count=4096] [seek=6144] dd: error writing '/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.00607695 s, 0.0 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0] [bs=1K] [count=10] [seek=10240] dd: error writing '/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.00490626 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 8192 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213* - 6213 - 2* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6213 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 8192 0 - 1 0 0 - lustre-MDT0000 1* - 1 - 1* - 0 - lustre-OST0000 66 - 1090 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 1090 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 66 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.0811149 s, 51.7 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Group quota in qpool1(soft limit:4 MB grace:60 seconds) -g quota_usr space:0 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.0929126 s, 45.1 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1] [bs=1K] [count=10] [seek=4096] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.151229 s, 67.7 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5124 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5124 8192 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 6148 - 1 - 0 - lustre-OST0001 0 - 41 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6189 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1] [bs=1K] [count=10] [seek=5120] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.121507 s, 84.3 kB/s Quota info for qpool1: Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6148* 4096 0 59s 0* 0 0 - Grace time is 59s Sleep through grace ... ...sleep 64 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6148 - 0 - 1 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 8192 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6148 - 7172 - 1 - 0 - lustre-OST0001 66* - 66 - 1* - 0 - Total allocated inode limit: 0, total allocated block limit: 7238 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1] [bs=1K] [count=4096] [seek=6144] dd: error writing '/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1': Disk quota exceeded 2+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0223179 s, 45.9 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1] [bs=1K] [count=10] [seek=10240] dd: error writing '/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.00384912 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7239 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7172 - 0 - 1 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7239 8192 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7172* - 7172 - 1* - 0 - lustre-OST0001 66* - 66 - 1* - 0 - Total allocated inode limit: 0, total allocated block limit: 7238 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 8192 0 - 1 0 0 - lustre-MDT0000 1* - 1 - 1* - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 66* - 66 - 1* - 0 - Total allocated inode limit: 0, total allocated block limit: 66 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.109649 s, 38.3 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Project quota in qpool1(soft:4 MB grace:60 sec) -p 1000 space:0 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.085628 s, 49.0 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2] [bs=1K] [count=10] [seek=4096] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.117292 s, 87.3 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5124 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5124 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5124 8192 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 6148 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6148 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2] [bs=1K] [count=10] [seek=5120] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.107739 s, 95.0 kB/s Quota info for qpool1: Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6148* 4096 0 59s 0* 0 0 - Grace time is 59s Sleep through grace ... ...sleep 64 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6149 8192 0 - 1 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 6148 - 7172 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 7172 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2] [bs=1K] [count=4096] [seek=6144] dd: error writing '/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2': Disk quota exceeded 2+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0207033 s, 49.5 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2] [bs=1K] [count=10] [seek=10240] dd: error writing '/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.00487012 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7239 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7237 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7239 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7237 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7173 8192 0 - 1 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 7172* - 7172 - 1* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 7172 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 66 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 66 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 8192 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2 Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.0928884 s, 45.2 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Destroy the created pools: qpool1 lustre.qpool1 oleg154-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg154-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg154-server: Pool lustre.qpool1 destroyed osc.lustre-OST0000-osc-ffff8cbe04bfb800.max_dirty_mb=462 osc.lustre-OST0001-osc-ffff8cbe04bfb800.max_dirty_mb=462 PASS 3b (384s) == sanity-quota test 3c: Quota pools: check block soft limit on different pools ========================================================== 11:23:26 (1776871406) limit 4 limit2 8 glbl_limit 12 grace1 70 grace2 60 glbl_grace 80 old_mdmb 462 mdmb 1 osc.lustre-OST0000-osc-ffff8cbe04bfb800.max_dirty_mb=1 osc.lustre-OST0001-osc-ffff8cbe04bfb800.max_dirty_mb=1 User quota in qpool2(soft:8 MB grace:60 seconds) -u quota_usr space:0 Creating new pool qpool1 oleg154-server: Pool lustre.qpool1 created Adding targets to pool oleg154-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg154-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 Waiting 90s for 'lustre-OST0000_UUID lustre-OST0001_UUID ' Creating new pool qpool2 oleg154-server: Pool lustre.qpool2 created Waiting 90s for '' Adding targets to pool oleg154-server: OST lustre-OST0000_UUID added to pool lustre.qpool2 oleg154-server: OST lustre-OST0001_UUID added to pool lustre.qpool2 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0] [count=8] 8+0 records in 8+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.232453 s, 36.1 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0] [bs=1K] [count=10] [seek=8192] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.127023 s, 80.6 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 9263 12288 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 9220 - 10244 - 1 - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 10328 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 9263 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 9220 - 0 - 1 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m20s; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0] [bs=1K] [count=10] [seek=9216] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.117747 s, 87.0 kB/s Quota info for qpool2: Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 10287* 8192 0 59s 0* 0 0 - Grace time is 59s Sleep through grace ... ...sleep 64 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 10289 12288 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 10244 - 11268 - 1 - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 11352 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 10289 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 10244 - 0 - 1 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m20s; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0] [bs=1K] [count=4096] [seek=10240] dd: error writing '/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0': Disk quota exceeded 2+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0190159 s, 53.8 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0] [bs=1K] [count=10] [seek=14336] dd: error writing '/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.00396563 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 11314 12288 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 11269* - 11269 - 1* - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 11353 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 11314 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 11269 - 0 - 1 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m20s; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 44 12288 0 - 1 0 0 - lustre-MDT0000 1* - 1 - 1* - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 84 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 44 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m20s; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0] [count=8] 8+0 records in 8+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.235842 s, 35.6 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Destroy the created pools: qpool1,qpool2 lustre.qpool1 oleg154-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg154-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg154-server: Pool lustre.qpool1 destroyed lustre.qpool2 oleg154-server: OST lustre-OST0000_UUID removed from pool lustre.qpool2 oleg154-server: OST lustre-OST0001_UUID removed from pool lustre.qpool2 oleg154-server: Pool lustre.qpool2 destroyed osc.lustre-OST0000-osc-ffff8cbe04bfb800.max_dirty_mb=462 osc.lustre-OST0001-osc-ffff8cbe04bfb800.max_dirty_mb=462 PASS 3c (149s) SKIP: sanity-quota test_4a skipping excluded test 4a == sanity-quota test 4b: Grace time strings handling ===== 11:25:55 (1776871555) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Valid grace strings test Block grace time: 1w3d; Inode grace time: 16m40s Block grace time: 5s; Inode grace time: 1w2d3h4m5s Invalid grace strings test lfs: bad inode-grace: 5c setquota failed: Unknown error -4 Set filesystem quotas. usage: setquota [-t] {-u|-U|-g|-G|-p|-P ID} {-b|-B|-i|-I LIMIT} [--lqa LQA_NAME] [--pool POOL] MOUNT_POINT setquota {-u|-g|-p ID} {--default|--delete} MOUNT_POINT lfs: bad inode-grace: 18446744073709551615 setquota failed: Unknown error -4 Set filesystem quotas. usage: setquota [-t] {-u|-U|-g|-G|-p|-P ID} {-b|-B|-i|-I LIMIT} [--lqa LQA_NAME] [--pool POOL] MOUNT_POINT setquota {-u|-g|-p ID} {--default|--delete} MOUNT_POINT lfs: bad inode-grace: -1 setquota failed: Unknown error -4 Set filesystem quotas. usage: setquota [-t] {-u|-U|-g|-G|-p|-P ID} {-b|-B|-i|-I LIMIT} [--lqa LQA_NAME] [--pool POOL] MOUNT_POINT setquota {-u|-g|-p ID} {--default|--delete} MOUNT_POINT PASS 4b (8s) == sanity-quota test 5: Chown & chgrp successfully even out of block/file quota ========================================================== 11:26:03 (1776871563) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Set quota limit (0 10M 0 1024) for quota_usr.quota_usr lfs setquota: warning: inode hardlimit '1024' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details lfs setquota: warning: inode hardlimit '1024' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details lfs setquota: warning: inode hardlimit '1024' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details -u quota_usr inodes:0 -g quota_usr inodes:0 -u quota_usr space:0 -g quota_usr space:0 -p 1000 inodes:0 -p 1000 space:0 Create more than 1024 files and more than 10 MB ... total: 1025 create in 1.14 seconds: 901.51 ops/second /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d5.sanity-quota/f5.sanity-quota-0_1 11+0 records in 11+0 records out 11534336 bytes (12 MB, 11 MiB) copied, 0.223496 s, 51.6 MB/s Chown files to quota_usr.quota_usr ... - unlinked 0 (time 1776871592 ; total 0 ; last 0) total: 1025 unlinks in 1 seconds: 1025.000000 unlinks/second Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 5 (51s) == sanity-quota test 6: Test dropping acquire request on master ========================================================== 11:26:54 (1776871614) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u quota_usr space:0 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_usr] [count=1] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0413629 s, 25.4 MB/s running as uid/gid/euid/egid 60001/60001/60001/60001, groups: 60001 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_2usr] [count=1] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200857 s, 52.2 MB/s at_max=20 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_usr] [count=3] [seek=1] [oflag=sync] [conv=notrunc] dd: error writing '/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_usr': Disk quota exceeded 2+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0707354 s, 14.8 MB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_val=601 fail_loc=0x513 old quota_slave timeout 10 osd-zfs.lustre-OST0000.quota_slave.timeout=10 osd-zfs.lustre-OST0001.quota_slave.timeout=10 running as uid/gid/euid/egid 60001/60001/60001/60001, groups: 60001 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_2usr] [count=3] [seek=1] [oflag=sync] [conv=notrunc] 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.101439 s, 31.0 MB/s Sleep for 41 seconds ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_usr] [count=3] [seek=1] [oflag=sync] [conv=notrunc] at_max=600 fail_val=0 fail_loc=0 dd: error writing '/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_usr': Disk quota exceeded 3+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 54.2305 s, 38.7 kB/s osd-zfs.lustre-OST0000.quota_slave.timeout=10 osd-zfs.lustre-OST0001.quota_slave.timeout=10 at_max=600 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 6 (107s) == sanity-quota test 7a: Quota reintegration (global index) ========================================================== 11:28:41 (1776871721) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u quota_usr space:0 Waiting 90s for 'none' -u quota_usr bhardlimit:0 Stop ost1... Stopping /mnt/lustre-ost1 (opts:) on oleg154-server Enable quota & set quota limit for quota_usr Waiting 90s for 'ugp' Updated after 2s: want 'ugp' got 'ugp' Start ost1... Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg154-server: oleg154-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg154-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg154-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg154-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg154-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg154-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg154-server: *.lustre-OST0001.recovery_status status: INACTIVE running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7a.sanity-quota/f7a.sanity-quota] [count=6] [oflag=sync] dd: error writing '/mnt/lustre/d7a.sanity-quota/f7a.sanity-quota': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.356247 s, 11.8 MB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Stop ost1... Stopping /mnt/lustre-ost1 (opts:) on oleg154-server Start ost1... Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg154-server: oleg154-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg154-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg154-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg154-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg154-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg154-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg154-server: *.lustre-OST0001.recovery_status status: INACTIVE running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7a.sanity-quota/f7a.sanity-quota] [count=6] [oflag=sync] 6+0 records in 6+0 records out 6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.462342 s, 13.6 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 7a (92s) == sanity-quota test 7b: Quota reintegration (slave index) ========================================================== 11:30:13 (1776871813) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u quota_usr space:0 Waiting 90s for 'none' Updated after 2s: want 'none' got 'none' -u quota_usr bhardlimit:0 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7b.sanity-quota/f7b.sanity-quota] [count=1] [oflag=sync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0772193 s, 13.6 MB/s fail_val=0 fail_loc=0xa02 Waiting 90s for 'ugp' running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7b.sanity-quota/f7b.sanity-quota] [count=1] [seek=1] [oflag=sync] [conv=notrunc] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0706634 s, 14.8 MB/s -u quota_usr bhardlimit:1026 fail_val=0 fail_loc=0 Restart ost to trigger reintegration... Stopping /mnt/lustre-ost1 (opts:) on oleg154-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg154-server: oleg154-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg154-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg154-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg154-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg154-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg154-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg154-server: *.lustre-OST0001.recovery_status status: INACTIVE -u quota_usr bhardlimit:16777216 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 7b (66s) == sanity-quota test 7c: Quota reintegration (restart mds during reintegration) ========================================================== 11:31:19 (1776871879) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u quota_usr space:0 Waiting 90s for 'none' fail_val=0 fail_loc=0xa03 Waiting 90s for 'ugp' osd-zfs.lustre-OST0000.quota_slave.force_reint=1 osd-zfs.lustre-OST0001.quota_slave.force_reint=1 Stop mds... Stopping /mnt/lustre-mds1 (opts:) on oleg154-server fail_val=0 fail_loc=0 Start mds... Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg154-server: oleg154-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg154-client: oleg154-server: ssh exited with exit code 1 Started lustre-MDT0000 affected facets: ost1 oleg154-server: oleg154-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg154-server: *.lustre-OST0000.recovery_status status: COMPLETE Waiting 200s for 'glb[1],slv[1],reint[0]' Waiting 190s for 'glb[1],slv[1],reint[0]'