-----============= acceptance-small: sanity-quota ============----- Wed Aug 26 08:27:00 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg118-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanity-quota.*ex': No such file or directory excepting tests: 2 4a 63 65 skipping tests SLOW=no: 61 12a 9 === sanity-quota: start setup 08:27:32 (1787747252) === oleg118-client.virtnet: executing check_config_client /mnt/lustre oleg118-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg118-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8ffb4c343000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8ffb4c343000.idle_timeout=debug oleg118-server: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanity-quota: finish setup 08:28:09 (1787747289) === using SAVE_PROJECT_SUPPORTED=0 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [true] running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [touch] [/mnt/lustre/d0_runas_test/f7647] running as uid/gid/euid/egid 60001/60001/60001/60001, groups: 60001 [true] running as uid/gid/euid/egid 60001/60001/60001/60001, groups: 60001 [touch] [/mnt/lustre/d0_runas_test/f7647] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 -u quota_usr space:0 -u quota_2usr space:0 -g quota_usr space:0 -g quota_2usr space:0 -p 1000 space:0 -p 1001 space:0 -u quota_usr inodes:0 -u quota_2usr inodes:0 -g quota_usr inodes:0 -g quota_2usr inodes:0 -p 1000 inodes:0 -p 1001 inodes:0 == sanity-quota test 0: Test basic quota performance ===== 08:29:18 (1787747358) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d0.sanity-quota/f0.sanity-quota-0] [count=10] [conv=fsync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 1.97059 s, 5.3 MB/s Waiting 90s for 'ugp' Updated after 2s: want 'ugp' got 'ugp' running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d0.sanity-quota/f0.sanity-quota-0] [count=10] [conv=fsync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 2.23747 s, 4.7 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 0 (68s) == sanity-quota test 1a: Block hard limit (normal use and out of quota) ========================================================== 08:30:26 (1787747426) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:10 MB) -u quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.32846 s, 3.9 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-0] [count=5] [seek=5] dd: error writing '/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-0': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 1.18769 s, 3.5 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.296529 s, 0.0 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u quota_usr space:0 -------------------------------------- Group quota (block hardlimit:10 MB) -g quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-1] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.53395 s, 3.4 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-1] [count=5] [seek=5] dd: error writing '/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-1': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 1.64485 s, 2.5 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-1] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-1': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.418966 s, 0.0 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -g quota_usr space:0 -p 1000 space:0 -------------------------------------- Project quota (block hardlimit:10 mb) /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-2 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-2] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.64933 s, 3.2 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-2] [count=5] [seek=5] dd: error writing '/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-2': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 1.34654 s, 3.1 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-2] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1a.sanity-quota/f1a.sanity-quota-2': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.252715 s, 0.0 kB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -p 1000 space:0 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1a (223s) == sanity-quota test 1b: Quota pools: Block hard limit (normal use and out of quota) ========================================================== 08:34:09 (1787747649) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:20 MB) Creating new pool qpool1 oleg118-server: Pool lustre.qpool1 created Adding targets to pool oleg118-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg118-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 -u quota_usr space:0 used 0 -u quota_usr bhardlimit:10240 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.749802 s, 7.0 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-0] [count=5] [seek=5] dd: error writing '/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-0': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.684576 s, 6.1 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.253141 s, 0.0 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u quota_usr space:0 debugfs 1.47.3-wc2 (11-Nov-2025) debugfs: No such file or directory while trying to open /tmp/lustre-mdt1 ls: Filesystem not open debugfs 1.47.3-wc2 (11-Nov-2025) debugfs: No such file or directory while trying to open /tmp/lustre-mdt1 ls: Filesystem not open -------------------------------------- Group quota (block hardlimit:20 MB) -g quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-1] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.22356 s, 4.3 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-1] [count=5] [seek=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.73416 s, 7.1 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-1] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-1': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.281579 s, 0.0 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -g quota_usr space:0 -p 1000 space:0 -------------------------------------- Project quota (block hardlimit:20 mb) /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-2 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-2] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.978628 s, 5.4 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-2] [count=5] [seek=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.620493 s, 8.4 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-2] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1b.sanity-quota/f1b.sanity-quota-2': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.350746 s, 0.0 kB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -p 1000 space:0 Destroy the created pools: qpool1 lustre.qpool1 oleg118-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg118-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg118-server: Pool lustre.qpool1 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1b (231s) == sanity-quota test 1c: Quota pools: check 3 pools with hardlimit only for global ========================================================== 08:38:00 (1787747880) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:20 MB) Creating new pool qpool1 oleg118-server: Pool lustre.qpool1 created Adding targets to pool oleg118-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg118-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 Creating new pool qpool2 oleg118-server: Pool lustre.qpool2 created Adding targets to pool oleg118-server: OST lustre-OST0000_UUID added to pool lustre.qpool2 oleg118-server: OST lustre-OST0001_UUID added to pool lustre.qpool2 -u quota_usr space:0 used 0 -u quota_usr bhardlimit:20480 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1c.sanity-quota/f1c.sanity-quota-0] [count=10] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 2.12154 s, 4.9 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1c.sanity-quota/f1c.sanity-quota-0] [count=10] [seek=10] dd: error writing '/mnt/lustre/d1c.sanity-quota/f1c.sanity-quota-0': Disk quota exceeded 10+0 records in 9+0 records out 9437184 bytes (9.4 MB, 9.0 MiB) copied, 2.12025 s, 4.5 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1c.sanity-quota/f1c.sanity-quota-0] [count=1] [seek=20] dd: error writing '/mnt/lustre/d1c.sanity-quota/f1c.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.303031 s, 0.0 kB/s -u quota_usr space:19462 qpool1 used 19462 -u quota_usr space:19462 qpool2 used 19462 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u quota_usr space:0 Destroy the created pools: qpool1,qpool2 lustre.qpool1 oleg118-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg118-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg118-server: Pool lustre.qpool1 destroyed lustre.qpool2 oleg118-server: OST lustre-OST0000_UUID removed from pool lustre.qpool2 oleg118-server: OST lustre-OST0001_UUID removed from pool lustre.qpool2 oleg118-server: Pool lustre.qpool2 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1c (152s) == sanity-quota test 1d: Quota pools: check block hardlimit on different pools ========================================================== 08:40:32 (1787748032) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:20 MB) Creating new pool qpool1 oleg118-server: Pool lustre.qpool1 created Adding targets to pool oleg118-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg118-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 Creating new pool qpool2 oleg118-server: Pool lustre.qpool2 created Adding targets to pool oleg118-server: OST lustre-OST0000_UUID added to pool lustre.qpool2 oleg118-server: OST lustre-OST0001_UUID added to pool lustre.qpool2 -u quota_usr space:0 used 0 -u quota_usr bhardlimit:20480 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1d.sanity-quota/f1d.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.789732 s, 6.6 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1d.sanity-quota/f1d.sanity-quota-0] [count=5] [seek=5] dd: error writing '/mnt/lustre/d1d.sanity-quota/f1d.sanity-quota-0': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.880172 s, 4.8 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1d.sanity-quota/f1d.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1d.sanity-quota/f1d.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.306732 s, 0.0 kB/s -u quota_usr space:9220 qpool1 used 9220 -u quota_usr space:9220 qpool2 used 9220 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u quota_usr space:0 Destroy the created pools: qpool1,qpool2 lustre.qpool1 oleg118-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg118-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg118-server: Pool lustre.qpool1 destroyed lustre.qpool2 oleg118-server: OST lustre-OST0000_UUID removed from pool lustre.qpool2 oleg118-server: OST lustre-OST0001_UUID removed from pool lustre.qpool2 oleg118-server: Pool lustre.qpool2 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1d (151s) == sanity-quota test 1e: Quota pools: global pool high block limit vs quota pool with small ========================================================== 08:43:03 (1787748183) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:53000000 MB) Creating new pool qpool1 oleg118-server: Pool lustre.qpool1 created Adding targets to pool oleg118-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 Waiting 90s for 'lustre-OST0001_UUID ' Updated after 2s: want 'lustre-OST0001_UUID ' got 'lustre-OST0001_UUID ' -u quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1e.sanity-quota/f1e.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.12514 s, 4.7 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1e.sanity-quota/f1e.sanity-quota-0] [count=5] [seek=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.21168 s, 4.3 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1e.sanity-quota/f1e.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1e.sanity-quota/f1e.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.270467 s, 0.0 kB/s Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1e.sanity-quota/f1e.sanity-quota-1] [count=20] 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 3.89259 s, 5.4 MB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u quota_usr space:0 Destroy the created pools: qpool1 lustre.qpool1 oleg118-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg118-server: Pool lustre.qpool1 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1e (130s) == sanity-quota test 1f: Quota pools: correct qunit after removing/adding OST ========================================================== 08:45:14 (1787748314) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:200 MB) Creating new pool qpool1 oleg118-server: Pool lustre.qpool1 created Adding targets to pool oleg118-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 -u quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.44586 s, 3.6 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0] [count=5] [seek=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.25137 s, 4.2 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.35051 s, 0.0 kB/s Removing lustre-OST0000_UUID from qpool1 oleg118-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 pdsh@oleg118-client: oleg118-server: ssh exited with exit code 1 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Adding targets to pool oleg118-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.6108 s, 3.3 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0] [count=5] [seek=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.11898 s, 4.7 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1f.sanity-quota//f1f.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.456862 s, 0.0 kB/s Destroy the created pools: qpool1 lustre.qpool1 oleg118-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg118-server: Pool lustre.qpool1 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 1f (156s) == sanity-quota test 1g: Quota pools: Block hard limit with wide striping ========================================================== 08:47:50 (1787748470) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 old_mdmb 462 mdmb 1 osc.lustre-OST0000-osc-ffff8ffb4c343000.max_dirty_mb=1 osc.lustre-OST0001-osc-ffff8ffb4c343000.max_dirty_mb=1 User quota (block hardlimit:40 MB) Creating new pool qpool1 oleg118-server: Pool lustre.qpool1 created Adding targets to pool oleg118-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg118-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 -u quota_usr space:0 used 0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1g.sanity-quota/f1g.sanity-quota-0] [count=10] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 8.41302 s, 1.2 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1g.sanity-quota/f1g.sanity-quota-0] [count=10] [seek=10] dd: error writing '/mnt/lustre/d1g.sanity-quota/f1g.sanity-quota-0': Disk quota exceeded 10+0 records in 9+0 records out 9437184 bytes (9.4 MB, 9.0 MiB) copied, 7.06727 s, 1.3 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1g.sanity-quota/f1g.sanity-quota-0] [count=2] [seek=20] [oflag=sync] dd: error writing '/mnt/lustre/d1g.sanity-quota/f1g.sanity-quota-0': Disk quota exceeded 2+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 1.35337 s, 775 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u quota_usr space:0 Destroy the created pools: qpool1 lustre.qpool1 oleg118-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg118-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg118-server: Pool lustre.qpool1 destroyed osc.lustre-OST0000-osc-ffff8ffb4c343000.max_dirty_mb=462 osc.lustre-OST0001-osc-ffff8ffb4c343000.max_dirty_mb=462 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1g (153s) == sanity-quota test 1h: Block hard limit test using fallocate ========================================================== 08:50:22 (1787748622) fallocate on zfs doesn't consume space fallocate not supported SKIP: sanity-quota test_1h need >= 2.13.57 and ldiskfs for fallocate SKIP 1h (5s) == sanity-quota test 1i: Quota pools: different limit and usage relations ========================================================== 08:50:28 (1787748628) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (block hardlimit:200 MB) Creating new pool qpool1 oleg118-server: Pool lustre.qpool1 created Adding targets to pool oleg118-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 Waiting 90s for 'lustre-OST0000_UUID ' -u quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.32566 s, 4.0 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0] [count=5] [seek=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.903064 s, 5.8 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.348743 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 10245 0 0 - 1 0 0 - Pool: lustre.qpool1 lustre-OST0000 10245* - 10245 - 1* - 0 - Total allocated inode limit: 0, total allocated block limit: 10245 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-1] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.10736 s, 4.7 MB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.71159 s, 3.1 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0] [count=5] [seek=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.978428 s, 5.4 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.27383 s, 0.0 kB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-1] [count=3] 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.830159 s, 3.8 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-2] [count=3] dd: error writing '/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-2': Disk quota exceeded 2+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.755223 s, 1.4 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-2] [seek=3] [count=1] dd: error writing '/mnt/lustre/d1i.sanity-quota//f1i.sanity-quota-2': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.301264 s, 0.0 kB/s Destroy the created pools: qpool1 lustre.qpool1 oleg118-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg118-server: Pool lustre.qpool1 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 1i (151s) == sanity-quota test 1j: Enable project quota enforcement for root ========================================================== 08:52:59 (1787748779) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -------------------------------------- Project quota (hardlimit: 40 mb, 2048 files) /home/green/git/lustre-release/lustre/utils/lfs project -sp 1000 /mnt/lustre/d1j.sanity-quota/test_dir /home/green/git/lustre-release/lustre/utils/lfs project -sp 1000 /mnt/lustre/d1j.sanity-quota/test_dir1 /home/green/git/lustre-release/lustre/utils/lfs project -sp 1001 /mnt/lustre/d1j.sanity-quota/test_dir2 /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-0 /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-1 /home/green/git/lustre-release/lustre/utils/lfs project -p 1001 /mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-2 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 40960 - 0 0 2048 - lustre-MDT0000 0 - 0 - 0 - 1024 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 1024, total allocated block limit: 0 Disk quotas for prj (pid 1001): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 81920 - 0 0 4096 - lustre-MDT0000 0 - 0 - 0 - 1024 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 1024, total allocated block limit: 0 running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir/tfile1-] [2048] - create 916 (time 1787748817.52 total 10.00 last 91.60) - create 1827 (time 1787748827.52 total 20.01 last 91.05) total: 2048 create in 22.83 seconds: 89.72 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir/tfile2-] [048] total: 4 create in 0.04 seconds: 97.55 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-0] [count=40] [oflag=direct] 40+0 records in 40+0 records out 41943040 bytes (42 MB, 40 MiB) copied, 4.74469 s, 8.8 MB/s running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-0] [count=20] [seek=40] [oflag=direct] 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 3.37655 s, 6.2 MB/s osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=1 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=1 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=1 error: get_param: param_path 'osd-*/*/quota_slave/root_prj_enable': No such file or directory Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 64768* 0 40960 - 2056* 0 2048 - lustre-MDT0000 3319 - 0 - 2056 - 2056 - lustre-OST0000 61449* - 61449 - 1* - 0 - lustre-OST0001 1 - 1025 - 1 - 0 - Total allocated inode limit: 2056, total allocated block limit: 62474 Disk quotas for prj (pid 1001): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 10 0 81920 - 2 0 4096 - lustre-MDT0000 9 - 0 - 2 - 1024 - lustre-OST0000 1 - 16384 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 1024, total allocated block limit: 16384 running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir1/tfile-] [2048] mknod(/mnt/lustre/d1j.sanity-quota/test_dir1/tfile-0) error: Disk quota exceeded total: 0 create in 0.03 seconds: 0.00 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-1] [count=40] [oflag=direct] dd: error writing '/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-1': Disk quota exceeded 2+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245601 s, 4.3 MB/s running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-1] [count=20] [seek=40] [oflag=direct] dd: error writing '/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-1': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.136037 s, 0.0 kB/s running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir2/tfile1-] [2048] - create 784 (time 1787748854.18 total 10.00 last 78.38) - create 1702 (time 1787748864.19 total 20.01 last 91.71) total: 2048 create in 23.76 seconds: 86.20 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir2/tfile2-] [048] total: 4 create in 0.06 seconds: 67.98 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-2] [count=40] [oflag=direct] 40+0 records in 40+0 records out 41943040 bytes (42 MB, 40 MiB) copied, 6.68971 s, 6.3 MB/s running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-2] [count=20] [seek=40] [oflag=direct] 20+0 records in 20+0 records out 20971520 bytes (21 MB, 20 MiB) copied, 3.48462 s, 6.0 MB/s osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=0 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=0 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=0 On MGS 192.168.201.118, active = nodemap.active=1 On MGS 192.168.201.118, default.admin_nodemap = nodemap.default.admin_nodemap=1 On MGS 192.168.201.118, default.trusted_nodemap = nodemap.default.trusted_nodemap=1 On MGS 192.168.201.118, test_1j.admin_nodemap = nodemap.test_1j.admin_nodemap=1 On MGS 192.168.201.118, test_1j.trusted_nodemap = nodemap.test_1j.trusted_nodemap=1 On MGS 192.168.201.118, test_1j.rbac = nodemap.test_1j.rbac=file_perms,dne_ops,quota_ops,byfid_ops,chlg_ops running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir/tfile3-] [2048] mknod(/mnt/lustre/d1j.sanity-quota/test_dir/tfile3-0) error: Disk quota exceeded total: 0 create in 0.03 seconds: 0.00 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-0] [count=0] [seek=] [oflag=direct] dd: invalid number: '' On MGS 192.168.201.118, active = nodemap.active=0 running as uid/gid/euid/egid 0/0/0/0, groups: 0 [createmany] [-m] [/mnt/lustre/d1j.sanity-quota/test_dir/tfile4-] [2048] - create 479 (time 1787748936.30 total 10.01 last 47.83) - create 1190 (time 1787748946.31 total 20.02 last 71.05) - create 1765 (time 1787748956.31 total 30.03 last 57.47) total: 2048 create in 34.87 seconds: 58.73 ops/second running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1j.sanity-quota/f1j.sanity-quota-0] [count=40] [seek=40] [oflag=direct] 40+0 records in 40+0 records out 41943040 bytes (42 MB, 40 MiB) copied, 6.58952 s, 6.4 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -p 1000 space:0 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=0 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=0 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-OST0001.quota_slave.root_prj_enable=0 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 1j (373s) == sanity-quota test 1k: LQA: Block hard limit (normal use and out of quota) ========================================================== 08:59:12 (1787749152) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 range_min 500, range_max 60100 Disk default usr quota: Filesystem bquota blimit bgrace iquota ilimit igrace /mnt/lustre 0 20480 - 0 0 - Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=10] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 1.82357 s, 5.8 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=10] [seek=10] dd: error writing '/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0': Disk quota exceeded 10+0 records in 9+0 records out 9437184 bytes (9.4 MB, 9.0 MiB) copied, 2.57316 s, 3.7 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=1] [seek=20] dd: error writing '/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.301899 s, 0.0 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=10] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 2.95894 s, 3.5 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=10] [seek=10] dd: error writing '/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0': Disk quota exceeded 10+0 records in 9+0 records out 9437184 bytes (9.4 MB, 9.0 MiB) copied, 2.27592 s, 4.1 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=1] [seek=20] dd: error writing '/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.395429 s, 0.0 kB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=10] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 2.12035 s, 4.9 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=10] [seek=10] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 2.37069 s, 4.4 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0] [count=1] [seek=20] dd: error writing '/mnt/lustre/d1k.sanity-quota/f1k.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.381089 s, 0.0 kB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 1k (169s) == sanity-quota test 1l: Async writes should not be rejected by quota with root_prj_enable ========================================================== 09:02:01 (1787749321) Starting client: oleg118-client.virtnet: -o user_xattr,flock 192.168.201.118@tcp:/lustre /mnt/lustre2 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 /home/green/git/lustre-release/lustre/utils/lfs project -sp 1000 /mnt/lustre/d1l.sanity-quota osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=1 osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=1 dd: error writing '/mnt/lustre/d1l.sanity-quota/f1l.sanity-quota.1': Disk quota exceeded 30+0 records in 29+0 records out 30408704 bytes (30 MB, 29 MiB) copied, 4.32685 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0821166 s, 12.8 MB/s osd-zfs.lustre-OST0000.quota_slave.root_prj_enable=0 osd-zfs.lustre-MDT0000.quota_slave.root_prj_enable=0 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete 192.168.201.118@tcp:/lustre /mnt/lustre2 lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg118-client.virtnet /mnt/lustre2 (opts:) PASS 1l (66s) SKIP: sanity-quota test_2 skipping excluded test 2 == sanity-quota test 3a: Block soft limit (start timer, timer goes off, stop timer) ========================================================== 09:03:09 (1787749389) old_mdmb 462 mdmb 1 osc.lustre-OST0000-osc-ffff8ffb4c343000.max_dirty_mb=1 osc.lustre-OST0001-osc-ffff8ffb4c343000.max_dirty_mb=1 User quota (soft limit:4 MB grace:60 seconds) -u quota_usr space:0 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.241372 s, 17.4 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0] [bs=1K] [count=10] [seek=4096] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.442301 s, 23.2 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5169* 4096 0 1m 2* 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 5124 - 6148 - 1 - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 6232 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5169 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 5124 - 0 - 1 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0] [bs=1K] [count=10] [seek=5120] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.388316 s, 26.4 kB/s Grace time is 59s Sleep through grace ... ...sleep 64 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6193* 4096 0 expired 2* 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6148 - 7172 - 1 - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 7256 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6193 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6148 - 0 - 1 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0] [bs=1K] [count=4096] [seek=6144] dd: error writing '/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0': Disk quota exceeded 2+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.160638 s, 6.4 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0] [bs=1K] [count=10] [seek=10240] dd: error writing '/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0424676 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7217* 4096 0 expired 2* 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7172* - 7172 - 1* - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 7256 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7217 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7172 - 0 - 1 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 44 4096 0 - 1 0 0 - lustre-MDT0000 1* - 1 - 1* - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 84 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 44 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-0] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.704728 s, 6.0 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Group quota (soft limit:4 MB grace:60 seconds) -g quota_usr space:0 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.562464 s, 7.5 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1] [bs=1K] [count=10] [seek=4096] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.678135 s, 15.1 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5129 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5129 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5129* 4096 0 59s 0* 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5129 - 6148 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6148 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1] [bs=1K] [count=10] [seek=5120] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.0132696 s, 772 kB/s Grace time is 59s Sleep through grace ... ...sleep 64 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215* 4096 0 expired 2* 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213* - 6213 - 2* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6213 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1] [bs=1K] [count=4096] [seek=6144] dd: error writing '/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0327924 s, 0.0 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1] [bs=1K] [count=10] [seek=10240] dd: error writing '/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0367444 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215* 4096 0 expired 2* 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213* - 6213 - 2* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6213 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 66 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 4096 0 - 1 0 0 - lustre-MDT0000 1* - 1 - 1* - 0 - lustre-OST0000 66 - 1090 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 1090 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-1] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.509823 s, 8.2 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Project quota (soft limit:4 MB grace:60 sec) -p 1000 space:0 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.677304 s, 6.2 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2] [bs=1K] [count=10] [seek=4096] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.615792 s, 16.6 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5129 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 0 - 1 - 0 - lustre-OST0001 6 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5129 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 0 - 1 - 0 - lustre-OST0001 6 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5124* 4096 0 59s 0* 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 6148 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6148 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2] [bs=1K] [count=10] [seek=5120] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.728756 s, 14.1 kB/s Grace time is 58s Sleep through grace ... ...sleep 63 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6148 - 0 - 1 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6148 - 0 - 1 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6149* 4096 0 expired 1* 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 6148 - 7172 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 7172 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2] [bs=1K] [count=4096] [seek=6144] dd: error writing '/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2': Disk quota exceeded 2+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.1627 s, 6.3 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2] [bs=1K] [count=10] [seek=10240] dd: error writing '/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0379629 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7239 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7172 - 0 - 1 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7239 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7172 - 0 - 1 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7173* 4096 0 expired 1* 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 7172* - 7172 - 1* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 7172 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 4096 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w Block grace time: 1m; Inode grace time: 1w /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2 Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3a.sanity-quota/f3a.sanity-quota-2] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.426499 s, 9.8 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete osc.lustre-OST0000-osc-ffff8ffb4c343000.max_dirty_mb=462 osc.lustre-OST0001-osc-ffff8ffb4c343000.max_dirty_mb=462 PASS 3a (515s) == sanity-quota test 3b: Quota pools: Block soft limit (start timer, expires, stop timer) ========================================================== 09:11:44 (1787749904) limit 4 glbl_limit 8 grace 60 glbl_grace 120 old_mdmb 462 mdmb 1 osc.lustre-OST0000-osc-ffff8ffb4c343000.max_dirty_mb=1 osc.lustre-OST0001-osc-ffff8ffb4c343000.max_dirty_mb=1 User quota in qpool1(soft limit:4 MB grace:60 seconds) -u quota_usr space:0 Creating new pool qpool1 oleg118-server: Pool lustre.qpool1 created Adding targets to pool oleg118-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg118-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.208697 s, 20.1 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0] [bs=1K] [count=10] [seek=4096] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.806932 s, 12.7 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5129 8192 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5129 - 6148 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6148 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5129 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5129 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0] [bs=1K] [count=10] [seek=5120] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.0064655 s, 1.6 MB/s Quota info for qpool1: Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5129* 4096 0 59s 0* 0 0 - Grace time is 59s Sleep through grace ... ...sleep 64 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 8192 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213* - 6213 - 2* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6213 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0] [bs=1K] [count=4096] [seek=6144] dd: error writing '/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0256103 s, 0.0 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0] [bs=1K] [count=10] [seek=10240] dd: error writing '/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0444812 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 8192 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213* - 6213 - 2* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6213 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 8192 0 - 1 0 0 - lustre-MDT0000 1* - 1 - 1* - 0 - lustre-OST0000 66 - 1090 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 1090 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 66 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-0] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.533464 s, 7.9 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Group quota in qpool1(soft limit:4 MB grace:60 seconds) -g quota_usr space:0 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.341933 s, 12.3 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1] [bs=1K] [count=10] [seek=4096] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.554316 s, 18.5 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5124 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5124 8192 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 6148 - 1 - 0 - lustre-OST0001 0 - 41 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6189 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1] [bs=1K] [count=10] [seek=5120] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.594992 s, 17.2 kB/s Quota info for qpool1: Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6148* 4096 0 59s 0* 0 0 - Grace time is 59s Sleep through grace ... ...sleep 64 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6148 - 0 - 1 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 8192 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6148 - 7172 - 1 - 0 - lustre-OST0001 66* - 66 - 1* - 0 - Total allocated inode limit: 0, total allocated block limit: 7238 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1] [bs=1K] [count=4096] [seek=6144] dd: error writing '/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1': Disk quota exceeded 2+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0772026 s, 13.3 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1] [bs=1K] [count=10] [seek=10240] dd: error writing '/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.026683 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7239 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7172 - 0 - 1 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7239 8192 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7172* - 7172 - 1* - 0 - lustre-OST0001 66* - 66 - 1* - 0 - Total allocated inode limit: 0, total allocated block limit: 7238 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 66 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 8192 0 - 1 0 0 - lustre-MDT0000 1* - 1 - 1* - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 66* - 66 - 1* - 0 - Total allocated inode limit: 0, total allocated block limit: 66 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-1] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.603958 s, 6.9 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Project quota in qpool1(soft:4 MB grace:60 sec) -p 1000 space:0 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.246749 s, 17.0 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2] [bs=1K] [count=10] [seek=4096] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.87706 s, 11.7 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5124 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5124 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 5124 8192 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 5124 - 6148 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 6148 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2] [bs=1K] [count=10] [seek=5120] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.707115 s, 14.5 kB/s Quota info for qpool1: Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6148* 4096 0 58s 0* 0 0 - Grace time is 58s Sleep through grace ... ...sleep 63 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6215 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 6213 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 6149 8192 0 - 1 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 6148 - 7172 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 7172 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2] [bs=1K] [count=4096] [seek=6144] dd: error writing '/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2': Disk quota exceeded 2+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.173363 s, 5.9 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2] [bs=1K] [count=10] [seek=10240] dd: error writing '/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0391001 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7239 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7237 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7239 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 7237 - 0 - 2 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 7173 8192 0 - 1 0 0 - lustre-MDT0000 2 - 0 - 1 - 0 - lustre-OST0000 7172* - 7172 - 1* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 7172 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 66 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 66 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 66 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 8192 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w Block grace time: 2m; Inode grace time: 1w /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2 Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3b.sanity-quota/f3b.sanity-quota-2] [count=4] 4+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.281067 s, 14.9 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Destroy the created pools: qpool1 lustre.qpool1 oleg118-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg118-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg118-server: Pool lustre.qpool1 destroyed osc.lustre-OST0000-osc-ffff8ffb4c343000.max_dirty_mb=462 osc.lustre-OST0001-osc-ffff8ffb4c343000.max_dirty_mb=462 PASS 3b (544s) == sanity-quota test 3c: Quota pools: check block soft limit on different pools ========================================================== 09:20:48 (1787750448) limit 4 limit2 8 glbl_limit 12 grace1 70 grace2 60 glbl_grace 80 old_mdmb 462 mdmb 1 osc.lustre-OST0000-osc-ffff8ffb4c343000.max_dirty_mb=1 osc.lustre-OST0001-osc-ffff8ffb4c343000.max_dirty_mb=1 User quota in qpool2(soft:8 MB grace:60 seconds) -u quota_usr space:0 Creating new pool qpool1 oleg118-server: Pool lustre.qpool1 created Adding targets to pool oleg118-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg118-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 Creating new pool qpool2 oleg118-server: Pool lustre.qpool2 created Adding targets to pool oleg118-server: OST lustre-OST0000_UUID added to pool lustre.qpool2 oleg118-server: OST lustre-OST0001_UUID added to pool lustre.qpool2 sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Write up to soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0] [count=8] 8+0 records in 8+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.492181 s, 17.0 MB/s Write to exceed soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0] [bs=1K] [count=10] [seek=8192] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.730767 s, 14.0 kB/s mmap write when over soft limit running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [multiop] [/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0.mmap] [OT40960SMW] Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 9265 12288 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 9220 - 10244 - 1 - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 10328 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 9265 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 9220 - 0 - 1 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m20s; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write before timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0] [bs=1K] [count=10] [seek=9216] 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 1.45618 s, 7.0 kB/s Quota info for qpool2: Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 10289* 8192 0 56s 2* 0 0 - Grace time is 56s Sleep through grace ... ...sleep 61 seconds Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 10289 12288 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 10244 - 11268 - 1 - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 11352 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 10289 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 10244 - 0 - 1 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m20s; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write after timer goes off running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0] [bs=1K] [count=4096] [seek=10240] dd: error writing '/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0': Disk quota exceeded 2+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.147077 s, 7.0 kB/s Write after cancel lru locks running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0] [bs=1K] [count=10] [seek=14336] dd: error writing '/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.0200525 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 11314 12288 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 11269* - 11269 - 1* - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 11353 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 11314 0 0 - 2 0 0 - lustre-MDT0000 2 - 0 - 2 - 0 - lustre-OST0000 11269 - 0 - 1 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m20s; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Unlink file to stop timer sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 44 12288 0 - 1 0 0 - lustre-MDT0000 1* - 1 - 1* - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 44 - 84 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 84 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 44 0 0 - 1 0 0 - lustre-MDT0000 1 - 0 - 1 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 44 - 0 - 1 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Block grace time: 1m20s; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Block grace time: 1w; Inode grace time: 1w Write ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d3c.sanity-quota/f3c.sanity-quota-0] [count=8] 8+0 records in 8+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 1.03199 s, 8.1 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Destroy the created pools: qpool1,qpool2 lustre.qpool1 oleg118-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg118-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg118-server: Pool lustre.qpool1 destroyed lustre.qpool2 oleg118-server: OST lustre-OST0000_UUID removed from pool lustre.qpool2 oleg118-server: OST lustre-OST0001_UUID removed from pool lustre.qpool2 oleg118-server: Pool lustre.qpool2 destroyed osc.lustre-OST0000-osc-ffff8ffb4c343000.max_dirty_mb=462 osc.lustre-OST0001-osc-ffff8ffb4c343000.max_dirty_mb=462 PASS 3c (216s) SKIP: sanity-quota test_4a skipping excluded test 4a == sanity-quota test 4b: Grace time strings handling ===== 09:24:27 (1787750667) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Valid grace strings test Block grace time: 1w3d; Inode grace time: 16m40s Block grace time: 5s; Inode grace time: 1w2d3h4m5s Invalid grace strings test lfs: bad inode-grace: 5c setquota failed: Unknown error -4 Set filesystem quotas. usage: setquota [-t] {-u|-U|-g|-G|-p|-P ID} {-b|-B|-i|-I LIMIT} [--lqa LQA_NAME] [--pool POOL] MOUNT_POINT setquota {-u|-g|-p ID} {--default|--delete} MOUNT_POINT lfs: bad inode-grace: 18446744073709551615 setquota failed: Unknown error -4 Set filesystem quotas. usage: setquota [-t] {-u|-U|-g|-G|-p|-P ID} {-b|-B|-i|-I LIMIT} [--lqa LQA_NAME] [--pool POOL] MOUNT_POINT setquota {-u|-g|-p ID} {--default|--delete} MOUNT_POINT lfs: bad inode-grace: -1 setquota failed: Unknown error -4 Set filesystem quotas. usage: setquota [-t] {-u|-U|-g|-G|-p|-P ID} {-b|-B|-i|-I LIMIT} [--lqa LQA_NAME] [--pool POOL] MOUNT_POINT setquota {-u|-g|-p ID} {--default|--delete} MOUNT_POINT PASS 4b (18s) == sanity-quota test 5: Chown & chgrp successfully even out of block/file quota ========================================================== 09:24:45 (1787750685) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Set quota limit (0 10M 0 1024) for quota_usr.quota_usr lfs setquota: warning: inode hardlimit '1024' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details lfs setquota: warning: inode hardlimit '1024' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details lfs setquota: warning: inode hardlimit '1024' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details -u quota_usr inodes:0 -g quota_usr inodes:0 -u quota_usr space:0 -g quota_usr space:0 -p 1000 inodes:0 -p 1000 space:0 Create more than 1024 files and more than 10 MB ... total: 1025 create in 8.47 seconds: 121.00 ops/second /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d5.sanity-quota/f5.sanity-quota-0_1 11+0 records in 11+0 records out 11534336 bytes (12 MB, 11 MiB) copied, 0.671493 s, 17.2 MB/s Chown files to quota_usr.quota_usr ... - unlinked 0 (time 1787750788 ; total 0 ; last 0) total: 1025 unlinks in 9 seconds: 113.888885 unlinks/second Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 5 (137s) == sanity-quota test 6: Test dropping acquire request on master ========================================================== 09:27:03 (1787750823) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u quota_usr space:0 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_usr] [count=1] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181235 s, 5.8 MB/s running as uid/gid/euid/egid 60001/60001/60001/60001, groups: 60001 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_2usr] [count=1] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.04416 s, 23.7 MB/s at_max=20 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_usr] [count=3] [seek=1] [oflag=sync] [conv=notrunc] dd: error writing '/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_usr': Disk quota exceeded 2+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.42182 s, 2.5 MB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_val=601 fail_loc=0x513 old quota_slave timeout 10 osd-zfs.lustre-OST0000.quota_slave.timeout=10 osd-zfs.lustre-OST0001.quota_slave.timeout=10 running as uid/gid/euid/egid 60001/60001/60001/60001, groups: 60001 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_2usr] [count=3] [seek=1] [oflag=sync] [conv=notrunc] 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.511966 s, 6.1 MB/s Sleep for 41 seconds ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_usr] [count=3] [seek=1] [oflag=sync] [conv=notrunc] at_max=600 fail_val=0 fail_loc=0 dd: error writing '/mnt/lustre/d6.sanity-quota/f6.sanity-quota-quota_usr': Disk quota exceeded 3+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 56.2135 s, 37.3 kB/s osd-zfs.lustre-OST0000.quota_slave.timeout=10 osd-zfs.lustre-OST0001.quota_slave.timeout=10 at_max=600 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 6 (161s) == sanity-quota test 7a: Quota reintegration (global index) ========================================================== 09:29:44 (1787750984) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u quota_usr space:0 -u quota_usr bhardlimit:0 Stop ost1... Stopping /mnt/lustre-ost1 (opts:) on oleg118-server Enable quota & set quota limit for quota_usr Start ost1... Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg118-server: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg118-client: oleg118-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg118-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg118-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg118-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg118-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg118-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg118-server: *.lustre-OST0001.recovery_status status: INACTIVE running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7a.sanity-quota/f7a.sanity-quota] [count=6] [oflag=sync] dd: error writing '/mnt/lustre/d7a.sanity-quota/f7a.sanity-quota': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 1.54188 s, 2.7 MB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Stop ost1... Stopping /mnt/lustre-ost1 (opts:) on oleg118-server Start ost1... Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg118-server: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg118-client: oleg118-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg118-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg118-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg118-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg118-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg118-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg118-server: *.lustre-OST0001.recovery_status status: INACTIVE running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7a.sanity-quota/f7a.sanity-quota] [count=6] [oflag=sync] 6+0 records in 6+0 records out 6291456 bytes (6.3 MB, 6.0 MiB) copied, 2.36691 s, 2.7 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 7a (245s) == sanity-quota test 7b: Quota reintegration (slave index) ========================================================== 09:33:49 (1787751229) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u quota_usr space:0 -u quota_usr bhardlimit:0 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7b.sanity-quota/f7b.sanity-quota] [count=1] [oflag=sync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241301 s, 4.3 MB/s fail_val=0 fail_loc=0xa02 Waiting 90s for 'ugp' Updated after 2s: want 'ugp' got 'ugp' running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7b.sanity-quota/f7b.sanity-quota] [count=1] [seek=1] [oflag=sync] [conv=notrunc] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197479 s, 5.3 MB/s -u quota_usr bhardlimit:1026 fail_val=0 fail_loc=0 Restart ost to trigger reintegration... Stopping /mnt/lustre-ost1 (opts:) on oleg118-server Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg118-server: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg118-client: oleg118-server: ssh exited with exit code 1 Started lustre-OST0000 affected facets: ost1 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg118-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg118-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg118-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg118-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg118-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg118-server: *.lustre-OST0001.recovery_status status: INACTIVE -u quota_usr bhardlimit:16777216 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 7b (158s) == sanity-quota test 7c: Quota reintegration (restart mds during reintegration) ========================================================== 09:36:27 (1787751387) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u quota_usr space:0 fail_val=0 fail_loc=0xa03 osd-zfs.lustre-OST0000.quota_slave.force_reint=1 osd-zfs.lustre-OST0001.quota_slave.force_reint=1 Stop mds... Stopping /mnt/lustre-mds1 (opts:) on oleg118-server fail_val=0 fail_loc=0 Start mds... Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg118-server: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg118-client: oleg118-server: ssh exited with exit code 1 Started lustre-MDT0000 affected facets: ost1 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg118-server: *.lustre-OST0000.recovery_status status: COMPLETE Waiting 200s for 'glb[1],slv[1],reint[0]' Waiting 150s for 'glb[1],slv[1],reint[0]' Waiting 140s for 'glb[1],slv[1],reint[0]' Updated after 85s: want 'glb[1],slv[1],reint[0]' got 'glb[1],slv[1],reint[0]' affected facets: ost2 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg118-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg118-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg118-server: *.lustre-OST0001.recovery_status status: INACTIVE affected facets: ost1 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg118-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg118-server: *.lustre-OST0001.recovery_status status: INACTIVE running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7c.sanity-quota/f7c.sanity-quota] [count=6] [oflag=sync] dd: error writing '/mnt/lustre/d7c.sanity-quota/f7c.sanity-quota': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 2.33893 s, 1.8 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 7c (226s) == sanity-quota test 7d: Quota reintegration (Transfer index in multiple bulks) ========================================================== 09:40:13 (1787751613) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 fail_val=0 fail_loc=0x608 Waiting 90s for 'u' Updated after 2s: want 'u' got 'u' affected facets: ost1 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0000.recovery_status 1475 oleg118-server: *.lustre-OST0000.recovery_status status: COMPLETE affected facets: ost2 oleg118-server: oleg118-server.virtnet: executing _wait_recovery_complete *.lustre-OST0001.recovery_status 1475 oleg118-server: *.lustre-OST0001.recovery_status status: INACTIVE fail_val=0 fail_loc=0 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7d.sanity-quota/f7d.sanity-quota] [count=21] [oflag=sync] dd: error writing '/mnt/lustre/d7d.sanity-quota/f7d.sanity-quota': Disk quota exceeded 20+0 records in 19+0 records out 19922944 bytes (20 MB, 19 MiB) copied, 6.27343 s, 3.2 MB/s running as uid/gid/euid/egid 60001/60001/60001/60001, groups: 60001 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7d.sanity-quota/f7d.sanity-quota-1] [count=21] [oflag=sync] dd: error writing '/mnt/lustre/d7d.sanity-quota/f7d.sanity-quota-1': Disk quota exceeded 20+0 records in 19+0 records out 19922944 bytes (20 MB, 19 MiB) copied, 8.766 s, 2.3 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 7d (100s) == sanity-quota test 7e: Quota reintegration (inode limits) ========================================================== 09:41:53 (1787751713) SKIP: sanity-quota test_7e needs >= 2 MDTs SKIP 7e (4s) == sanity-quota test 7f: Quota reintegration automatically ========================================================== 09:41:57 (1787751717) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 1048576 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 fail_loc=0xa11 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d7f.sanity-quota/f7f.sanity-quota-0] [count=1024] [oflag=direct] dd: error writing '/mnt/lustre/d7f.sanity-quota/f7f.sanity-quota-0': Disk quota exceeded 1024+0 records in 1023+0 records out 1072693248 bytes (1.1 GB, 1023 MiB) copied, 157.64 s, 6.8 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 7f (226s) == sanity-quota test 8: Run dbench with quota enabled ==== 09:45:43 (1787751943) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Set enough high limit for user: quota_usr Set enough high limit for group: quota_usr /home/green/git/lustre-release/lustre/utils/lfs project -sp 1000 /mnt/lustre/d8.sanity-quota Set enough high limit for project: 1000 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 0 [bash] [rundbench] [-D] [/mnt/lustre/d8.sanity-quota] [3] [-t] [120] looking for dbench program /usr/bin/dbench found dbench client file /usr/share/dbench/client.txt '/usr/share/dbench/client.txt' -> 'client.txt' running 'dbench 3 -t 120' on /mnt/lustre/d8.sanity-quota at Wed Aug 26 09:46:12 EDT 2026 waiting for dbench pid 92880 dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 Running for 120 seconds with load 'client.txt' and minimum warmup 24 secs failed to create barrier semaphore 0 of 3 processes prepared for launch 0 sec 3 of 3 processes prepared for launch 0 sec releasing clients 1 14 0.39 MB/sec warmup 1 sec latency 166.572 ms 3 53 3.26 MB/sec warmup 2 sec latency 237.130 ms 3 90 4.13 MB/sec warmup 3 sec latency 261.596 ms 3 121 4.08 MB/sec warmup 4 sec latency 244.048 ms 3 148 3.76 MB/sec warmup 5 sec latency 253.924 ms 3 179 3.61 MB/sec warmup 6 sec latency 209.032 ms 3 216 3.60 MB/sec warmup 7 sec latency 171.110 ms 3 277 3.89 MB/sec warmup 8 sec latency 125.626 ms 3 308 3.73 MB/sec warmup 9 sec latency 195.455 ms 3 340 3.67 MB/sec warmup 10 sec latency 156.290 ms 3 384 3.71 MB/sec warmup 11 sec latency 146.448 ms 3 421 3.70 MB/sec warmup 12 sec latency 201.602 ms 3 466 3.75 MB/sec warmup 13 sec latency 122.394 ms 3 507 3.76 MB/sec warmup 14 sec latency 137.301 ms 3 552 3.83 MB/sec warmup 15 sec latency 188.232 ms 3 643 3.94 MB/sec warmup 16 sec latency 144.796 ms 3 681 3.73 MB/sec warmup 17 sec latency 183.146 ms 3 719 3.54 MB/sec warmup 18 sec latency 193.368 ms 3 756 3.36 MB/sec warmup 19 sec latency 223.855 ms 3 792 3.21 MB/sec warmup 20 sec latency 226.394 ms 3 819 3.06 MB/sec warmup 21 sec latency 221.602 ms 3 877 2.95 MB/sec warmup 22 sec latency 236.633 ms 3 911 2.83 MB/sec warmup 23 sec latency 214.715 ms 3 975 0.23 MB/sec execute 1 sec latency 285.007 ms 3 1011 0.98 MB/sec execute 2 sec latency 356.013 ms 3 1091 1.14 MB/sec execute 3 sec latency 108.173 ms 3 1146 0.90 MB/sec execute 4 sec latency 151.127 ms 3 1195 0.74 MB/sec execute 5 sec latency 175.129 ms 3 1243 0.65 MB/sec execute 6 sec latency 138.061 ms 3 1348 1.39 MB/sec execute 7 sec latency 205.598 ms 3 1422 1.58 MB/sec execute 8 sec latency 132.276 ms 3 1474 1.41 MB/sec execute 9 sec latency 243.433 ms 3 1527 1.28 MB/sec execute 10 sec latency 365.940 ms 3 1573 1.18 MB/sec execute 11 sec latency 155.448 ms 3 1619 1.09 MB/sec execute 12 sec latency 262.853 ms 3 1659 1.02 MB/sec execute 13 sec latency 191.514 ms 3 1696 0.95 MB/sec execute 14 sec latency 150.072 ms 3 1754 0.89 MB/sec execute 15 sec latency 148.452 ms 3 1809 0.85 MB/sec execute 16 sec latency 136.868 ms 3 1845 0.80 MB/sec execute 17 sec latency 210.208 ms 3 1879 0.76 MB/sec execute 18 sec latency 386.710 ms 3 1942 0.73 MB/sec execute 19 sec latency 196.337 ms 3 1986 0.70 MB/sec execute 20 sec latency 121.894 ms 3 2012 0.67 MB/sec execute 21 sec latency 197.881 ms 3 2068 0.66 MB/sec execute 22 sec latency 158.073 ms 3 2095 0.63 MB/sec execute 23 sec latency 265.026 ms 3 2138 0.61 MB/sec execute 24 sec latency 235.831 ms 3 2166 0.59 MB/sec execute 25 sec latency 137.064 ms 3 2210 0.57 MB/sec execute 26 sec latency 128.276 ms 3 2287 0.62 MB/sec execute 27 sec latency 87.881 ms 3 2341 0.65 MB/sec execute 28 sec latency 418.706 ms 3 2361 0.63 MB/sec execute 29 sec latency 473.211 ms 3 2391 0.62 MB/sec execute 30 sec latency 232.980 ms 3 2413 0.61 MB/sec execute 31 sec latency 387.833 ms 3 2447 0.59 MB/sec execute 32 sec latency 272.611 ms 3 2489 0.63 MB/sec execute 33 sec latency 225.354 ms 3 2581 0.63 MB/sec execute 34 sec latency 105.535 ms 3 2677 0.73 MB/sec execute 35 sec latency 127.038 ms 3 2809 0.85 MB/sec execute 36 sec latency 264.990 ms 3 2909 0.92 MB/sec execute 37 sec latency 264.046 ms 3 2986 0.92 MB/sec execute 38 sec latency 170.241 ms 3 3064 0.92 MB/sec execute 39 sec latency 296.279 ms 3 3194 0.93 MB/sec execute 40 sec latency 350.310 ms 3 3281 0.97 MB/sec execute 41 sec latency 250.301 ms 3 3360 0.97 MB/sec execute 42 sec latency 141.827 ms 3 3486 1.09 MB/sec execute 43 sec latency 82.988 ms 3 3571 1.13 MB/sec execute 44 sec latency 114.097 ms 3 3645 1.12 MB/sec execute 45 sec latency 91.704 ms 3 3688 1.10 MB/sec execute 46 sec latency 169.549 ms 3 3732 1.10 MB/sec execute 47 sec latency 131.746 ms 3 3775 1.12 MB/sec execute 48 sec latency 163.484 ms 3 3810 1.10 MB/sec execute 49 sec latency 123.267 ms 3 3853 1.08 MB/sec execute 50 sec latency 117.618 ms 3 3898 1.07 MB/sec execute 51 sec latency 136.822 ms 3 3947 1.05 MB/sec execute 52 sec latency 128.672 ms 3 4003 1.03 MB/sec execute 53 sec latency 94.451 ms 3 4056 1.01 MB/sec execute 54 sec latency 92.116 ms 3 4102 0.99 MB/sec execute 55 sec latency 116.503 ms 3 4139 0.98 MB/sec execute 56 sec latency 176.128 ms 3 4191 0.96 MB/sec execute 57 sec latency 150.980 ms 3 4227 0.95 MB/sec execute 58 sec latency 176.188 ms 3 4246 0.94 MB/sec execute 59 sec latency 422.869 ms 3 4275 0.92 MB/sec execute 60 sec latency 142.783 ms 3 4301 0.91 MB/sec execute 61 sec latency 374.039 ms 3 4346 0.90 MB/sec execute 62 sec latency 94.788 ms 3 4374 0.89 MB/sec execute 63 sec latency 202.032 ms 3 4435 0.88 MB/sec execute 64 sec latency 132.551 ms 3 4466 0.88 MB/sec execute 65 sec latency 182.677 ms 3 4490 0.87 MB/sec execute 66 sec latency 279.472 ms 3 4537 0.85 MB/sec execute 67 sec latency 111.767 ms 3 4565 0.85 MB/sec execute 68 sec latency 221.974 ms 3 4652 0.88 MB/sec execute 69 sec latency 119.544 ms 3 4717 0.87 MB/sec execute 70 sec latency 88.782 ms 3 4759 0.86 MB/sec execute 71 sec latency 99.627 ms 3 4808 0.85 MB/sec execute 72 sec latency 83.121 ms 3 4852 0.84 MB/sec execute 73 sec latency 177.708 ms 3 4965 0.94 MB/sec execute 74 sec latency 130.558 ms 3 5010 0.93 MB/sec execute 75 sec latency 117.249 ms 3 5060 0.92 MB/sec execute 76 sec latency 244.164 ms 3 5123 0.91 MB/sec execute 77 sec latency 89.223 ms 3 5156 0.90 MB/sec execute 78 sec latency 130.167 ms 3 5211 0.89 MB/sec execute 79 sec latency 243.214 ms 3 5256 0.88 MB/sec execute 80 sec latency 127.906 ms 3 5289 0.87 MB/sec execute 81 sec latency 117.127 ms 3 5348 0.86 MB/sec execute 82 sec latency 100.002 ms 3 5388 0.85 MB/sec execute 83 sec latency 103.924 ms 3 5446 0.84 MB/sec execute 84 sec latency 91.198 ms 3 5510 0.84 MB/sec execute 85 sec latency 213.711 ms 3 5562 0.83 MB/sec execute 86 sec latency 115.522 ms 3 5598 0.82 MB/sec execute 87 sec latency 113.228 ms 3 5680 0.82 MB/sec execute 88 sec latency 106.603 ms 3 5725 0.81 MB/sec execute 89 sec latency 144.226 ms 3 5780 0.80 MB/sec execute 90 sec latency 110.007 ms 3 5888 0.83 MB/sec execute 91 sec latency 69.556 ms 3 5924 0.82 MB/sec execute 92 sec latency 338.654 ms 3 5958 0.81 MB/sec execute 93 sec latency 176.637 ms 3 5989 0.81 MB/sec execute 94 sec latency 297.831 ms 3 6035 0.82 MB/sec execute 95 sec latency 107.884 ms 3 6124 0.82 MB/sec execute 96 sec latency 258.923 ms 3 6158 0.81 MB/sec execute 97 sec latency 138.743 ms 3 6322 0.89 MB/sec execute 98 sec latency 165.051 ms 3 6450 0.91 MB/sec execute 99 sec latency 176.818 ms 3 6485 0.92 MB/sec execute 100 sec latency 269.363 ms 3 6554 0.91 MB/sec execute 101 sec latency 311.256 ms 3 6741 0.93 MB/sec execute 102 sec latency 169.154 ms 3 6829 0.95 MB/sec execute 103 sec latency 226.889 ms 3 6920 0.94 MB/sec execute 104 sec latency 107.941 ms 3 7058 1.02 MB/sec execute 105 sec latency 126.938 ms 3 7121 1.01 MB/sec execute 106 sec latency 95.035 ms 3 7193 1.01 MB/sec execute 107 sec latency 82.470 ms 3 7238 1.00 MB/sec execute 108 sec latency 147.623 ms 3 7294 1.01 MB/sec execute 109 sec latency 142.077 ms 3 7341 1.01 MB/sec execute 110 sec latency 131.138 ms 3 7376 1.00 MB/sec execute 111 sec latency 186.631 ms 3 7424 0.99 MB/sec execute 112 sec latency 90.219 ms 3 7462 0.99 MB/sec execute 113 sec latency 153.912 ms 3 7502 0.98 MB/sec execute 114 sec latency 137.527 ms 3 7542 0.97 MB/sec execute 115 sec latency 114.864 ms 3 7579 0.96 MB/sec execute 116 sec latency 126.805 ms 3 7613 0.96 MB/sec execute 117 sec latency 105.163 ms 3 7654 0.95 MB/sec execute 118 sec latency 136.322 ms 3 7697 0.94 MB/sec execute 119 sec latency 129.786 ms 3 cleanup 120 sec 3 cleanup 121 sec 3 cleanup 122 sec 3 cleanup 123 sec 3 cleanup 124 sec 1 cleanup 125 sec 0 cleanup 125 sec Operation Count AvgLat MaxLat ---------------------------------------- NTCreateX 3520 50.058 265.008 Close 2561 12.510 80.036 Rename 147 76.130 205.572 Unlink 721 28.048 129.762 Qpathinfo 3344 13.557 216.519 Qfileinfo 558 1.795 29.328 Qfsinfo 575 8.236 165.989 Sfileinfo 261 55.266 125.017 Find 1235 4.917 176.098 WriteX 1695 11.116 143.193 ReadX 5608 0.073 14.330 LockX 12 19.961 58.375 UnlockX 12 11.230 24.743 Flush 192 153.734 473.189 Throughput 0.941151 MB/sec 3 clients 3 procs max_latency=473.211 ms stopping dbench on /mnt/lustre/d8.sanity-quota at Wed Aug 26 09:48:42 EDT 2026 with return code 0 clean dbench files on /mnt/lustre/d8.sanity-quota /mnt/lustre/d8.sanity-quota /mnt/lustre/d8.sanity-quota removed directory 'clients/client2/~dmtmp/SEED' removed directory 'clients/client2/~dmtmp/PM' removed directory 'clients/client2/~dmtmp/WORDPRO' removed directory 'clients/client2/~dmtmp/PWRPNT' removed directory 'clients/client2/~dmtmp/EXCEL' removed directory 'clients/client2/~dmtmp/WORD' removed directory 'clients/client2/~dmtmp/PARADOX' removed directory 'clients/client2/~dmtmp/ACCESS' removed directory 'clients/client2/~dmtmp/COREL' removed directory 'clients/client2/~dmtmp' removed directory 'clients/client2' removed directory 'clients/client1/~dmtmp/EXCEL' removed directory 'clients/client1/~dmtmp/PWRPNT' removed directory 'clients/client1/~dmtmp/PARADOX' removed directory 'clients/client1/~dmtmp/SEED' removed directory 'clients/client1/~dmtmp/ACCESS' removed directory 'clients/client1/~dmtmp/PM' removed directory 'clients/client1/~dmtmp/WORDPRO' removed directory 'clients/client1/~dmtmp/WORD' removed directory 'clients/client1/~dmtmp/COREL' removed directory 'clients/client1/~dmtmp' removed directory 'clients/client1' removed directory 'clients/client0/~dmtmp/EXCEL' removed directory 'clients/client0/~dmtmp/PM' removed directory 'clients/client0/~dmtmp/WORDPRO' removed directory 'clients/client0/~dmtmp/PWRPNT' removed directory 'clients/client0/~dmtmp/SEED' removed directory 'clients/client0/~dmtmp/COREL' removed directory 'clients/client0/~dmtmp/ACCESS' removed directory 'clients/client0/~dmtmp/WORD' removed directory 'clients/client0/~dmtmp/PARADOX' removed directory 'clients/client0/~dmtmp' removed directory 'clients/client0' removed directory 'clients' removed 'client.txt' /mnt/lustre/d8.sanity-quota dbench successfully finished /home/green/git/lustre-release/lustre/utils/lfs project -C /mnt/lustre/d8.sanity-quota Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 8 (223s) SKIP: sanity-quota test_9 skipping SLOW test 9 == sanity-quota test 10: Test quota for root user ======== 09:49:27 (1787752167) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 lfs setquota: can't set quota for root usr/group/project. setquota failed: Operation not permitted lfs setquota: can't set quota for root usr/group/project. setquota failed: Operation not permitted lfs setquota: can't set quota for root usr/group/project. setquota failed: Operation not permitted Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 2048 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 -u quota_usr space:0 running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d10.sanity-quota/f10.sanity-quota] [count=3] [oflag=sync] 3+0 records in 3+0 records out 3145728 bytes (3.1 MB, 3.0 MiB) copied, 0.939877 s, 3.3 MB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 10 (62s) == sanity-quota test 11: Chown/chgrp ignores quota ======= 09:50:30 (1787752230) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 lfs setquota: warning: inode hardlimit '1' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 1 - lustre-MDT0000 0 - 0 - 0 - 1 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 1, total allocated block limit: 0 -u quota_usr inodes:2 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 11 (56s) SKIP: sanity-quota test_12a skipping SLOW test 12a == sanity-quota test 12b: Inode quota rebalancing ======== 09:51:28 (1787752288) SKIP: sanity-quota test_12b needs >= 2 MDTs SKIP 12b (5s) == sanity-quota test 13: Cancel per-ID lock in the LRU list ========================================================== 09:51:33 (1787752293) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 0 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 -u quota_usr space:0 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d13.sanity-quota/f13.sanity-quota] [count=1] [oflag=sync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.556372 s, 1.9 MB/s -u quota_usr bhardlimit:0 -u quota_usr space:1026 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 13 (80s) == sanity-quota test 14: check panic in qmt_site_recalc_cb ========================================================== 09:52:53 (1787752373) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Creating new pool qpool1 oleg118-server: Pool lustre.qpool1 created Adding targets to pool oleg118-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 Waiting 90s for 'lustre-OST0000_UUID ' Updated after 2s: want 'lustre-OST0000_UUID ' got 'lustre-OST0000_UUID ' running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d14.sanity-quota/f14.sanity-quota-0] [count=10] [oflag=direct] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 1.4807 s, 7.1 MB/s Stop ost1... Stopping /mnt/lustre-ost1 (opts:) on oleg118-server Removing lustre-OST0000_UUID from qpool1 oleg118-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 pdsh@oleg118-client: oleg118-server: ssh exited with exit code 1 Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg118-server: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg118-client: oleg118-server: ssh exited with exit code 1 Started lustre-OST0000 Destroy the created pools: qpool1 lustre.qpool1 oleg118-server: Pool lustre.qpool1 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 14 (98s) == sanity-quota test 15: Set over 4T block quota ========= 09:54:31 (1787752471) sleep 5 for ZFS MDS Waiting for MDT destroys to complete -u quota_usr bhardlimit:26388279066624 -g quota_usr bhardlimit:26388279066624 PASS 15 (34s) == sanity-quota test 16a: lfs quota should skip the inactive MDT/OST ========================================================== 09:55:06 (1787752506) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d16a.sanity-quota/f16a.sanity-quota] [count=50] 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 7.99088 s, 6.6 MB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 23557 0 512000 - 1 0 10240 - Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 1 0 512000 - 1 0 10240 - lustre-MDT0000 1 - 0 - 1 - 4096 - lustre-OST0000 [0] - [0] - [0] - [0] - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 4096, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 1 0 512000 - 1 0 10240 - lustre-MDT0000 1 - 0 - 1 - 4096 - lustre-OST0000 [0] - [0] - [0] - [0] - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 4096, total allocated block limit: 0 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 16a (57s) == sanity-quota test 16b: lfs quota should skip the nonexistent MDT/OST ========================================================== 09:56:03 (1787752563) SKIP: sanity-quota test_16b needs >= 3 MDTs SKIP 16b (4s) == sanity-quota test 17: DQACQ return recoverable error == 09:56:07 (1787752567) DQACQ return -ENOLCK sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u quota_usr space:0 Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 10240 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 -u quota_usr space:0 fail_val=37 fail_loc=0xa04 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d17.sanity-quota/f17.sanity-quota] [count=1] [oflag=direct] fail_val=0 fail_loc=0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 7.33234 s, 143 kB/s -u quota_usr space:1026 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete DQACQ return -EAGAIN sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u quota_usr space:0 Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 10240 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 -u quota_usr space:0 fail_val=11 fail_loc=0xa04 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d17.sanity-quota/f17.sanity-quota] [count=1] [oflag=direct] fail_val=0 fail_loc=0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 6.76273 s, 155 kB/s -u quota_usr space:1026 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete DQACQ return -ETIMEDOUT sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u quota_usr space:0 Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 10240 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 -u quota_usr space:0 fail_val=110 fail_loc=0xa04 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d17.sanity-quota/f17.sanity-quota] [count=1] [oflag=direct] fail_val=0 fail_loc=0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 6.54549 s, 160 kB/s -u quota_usr space:1026 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete DQACQ return -ENOTCONN sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u quota_usr space:0 Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 10240 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 -u quota_usr space:0 fail_val=107 fail_loc=0xa04 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d17.sanity-quota/f17.sanity-quota] [count=1] [oflag=direct] fail_val=0 fail_loc=0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.51663 s, 298 kB/s -u quota_usr space:1026 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 17 (379s) == sanity-quota test 18: MDS failover while writing, no watchdog triggered (b14840) ========================================================== 10:02:27 (1787752947) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (limit: 200) Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 204800 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 -u quota_usr space:0 Write 100M (buffered) ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d18.sanity-quota/f18.sanity-quota] [count=100] UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210304 2688 2205568 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3770368 2048 3756032 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 2048 3767296 1% /mnt/lustre[OST:1] filesystem_summary: 7541760 4096 7523328 1% /mnt/lustre Fail mds for 40 seconds Failing mds1 on oleg118-server Stopping /mnt/lustre-mds1 (opts:) on oleg118-server 10:03:08 (1787752988) shut down facet: mds1 facet_host: oleg118-server facet_failover_host: oleg118-server Failover mds1 to oleg118-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg118-server: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg118-client: oleg118-server: ssh exited with exit code 1 Started lustre-MDT0000 10:03:40 (1787753020) targets are mounted 10:03:40 (1787753020) facet_failover done oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 56.6381 s, 1.9 MB/s (dd_pid=113972, time=0, timeout=600) Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 102413 0 204800 - 1 0 0 - lustre-MDT0000 2* - 2 - 1* - 0 - lustre-OST0000 102412 - 114688 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 114688 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 User quota (limit: 200) Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 0 204800 - 0 0 0 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 -u quota_usr space:0 Write 100M (directio) ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d18.sanity-quota/f18.sanity-quota] [count=100] [oflag=direct] UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210176 2432 2205696 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3770368 2048 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 2048 3767296 1% /mnt/lustre[OST:1] filesystem_summary: 7541760 4096 7528448 1% /mnt/lustre Fail mds for 40 seconds Failing mds1 on oleg118-server Stopping /mnt/lustre-mds1 (opts:) on oleg118-server 10:05:10 (1787753110) shut down facet: mds1 facet_host: oleg118-server facet_failover_host: oleg118-server Failover mds1 to oleg118-server mount facets: mds1 Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg118-server: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg118-client: oleg118-server: ssh exited with exit code 1 Started lustre-MDT0000 10:05:47 (1787753147) targets are mounted 10:05:47 (1787753147) facet_failover done oleg118-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 68.9593 s, 1.5 MB/s (dd_pid=116372, time=6, timeout=600) Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 102414 0 204800 - 1 0 0 - lustre-MDT0000 2* - 2 - 1* - 0 - lustre-OST0000 102413 - 114688 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 114688 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 18 (301s) == sanity-quota test 19: Updating admin limits doesn't zero operational limits(b14790) ========================================================== 10:07:28 (1787753248) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Set user quota (limit: 5M) Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 2 0 5120 - 1 0 0 - lustre-MDT0000 2* - 2 - 1* - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 -u quota_usr space:2 Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 2 0 5120 - 1 0 0 - lustre-MDT0000 2* - 2 - 1* - 0 - lustre-OST0000 1 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Files for user (quota_usr), count=1: File: /mnt/lustre/d19.sanity-quota/f19.sanity-quota Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272510625 Links: 1 Access: (0644/-rw-r--r--) Uid: (60000/quota_usr) Gid: (60000/quota_usr) Access: 2026-08-26 10:07:49.000000000 -0400 Modify: 2026-08-26 10:07:49.000000000 -0400 Change: 2026-08-26 10:07:49.000000000 -0400 Birth: 2026-08-26 10:07:49.000000000 -0400 Block quota isn't 0 (u:quota_usr:2). Update quota limits Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 2 0 5120 - 1 0 0 - lustre-MDT0000 2* - 2 - 1* - 0 - lustre-OST0000 1 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 -u quota_usr space:2 Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 2 0 5120 - 1 0 0 - lustre-MDT0000 2* - 2 - 1* - 0 - lustre-OST0000 1 - 0 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Files for user (quota_usr), count=1: File: /mnt/lustre/d19.sanity-quota/f19.sanity-quota Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272510625 Links: 1 Access: (0644/-rw-r--r--) Uid: (60000/quota_usr) Gid: (60000/quota_usr) Access: 2026-08-26 10:07:49.000000000 -0400 Modify: 2026-08-26 10:07:49.000000000 -0400 Change: 2026-08-26 10:07:49.000000000 -0400 Birth: 2026-08-26 10:07:49.000000000 -0400 Block quota isn't 0 (u:quota_usr:2). running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d19.sanity-quota/f19.sanity-quota] [count=6] dd: error writing '/mnt/lustre/d19.sanity-quota/f19.sanity-quota': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 1.27964 s, 3.3 MB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 4101 0 5120 - 1 0 0 - lustre-MDT0000 2* - 2 - 1* - 0 - lustre-OST0000 4100 - 5118 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 5118 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d19.sanity-quota/f19.sanity-quota] [count=6] [seek=6] dd: error writing '/mnt/lustre/d19.sanity-quota/f19.sanity-quota': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.292593 s, 0.0 kB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 4101 0 5120 - 1 0 0 - lustre-MDT0000 2* - 2 - 1* - 0 - lustre-OST0000 4100 - 5118 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 5118 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 19 (83s) == sanity-quota test 20: Test if setquota specifiers work properly (b15754) ========================================================== 10:08:51 (1787753331) -u quota_usr bsoftlimit:2097152 -u quota_usr bhardlimit:1073741824 -u quota_usr isoftlimit:4096 -u quota_usr ihardlimit:3145728 PASS 20 (33s) == sanity-quota test 21: Setquota while writing & deleting (b16053) ========================================================== 10:09:24 (1787753364) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Set limit(block:10G; file:1000000) for user: quota_usr Set limit(block:10G; file:1000000) for group: quota_usr lfs setquota: warning: block hardlimit '10' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details Set limit(block:10G; file:) for project: 1000 lfs setquota: warning: block hardlimit '10' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details Set quota for 1 times Set quota for 2 times Set quota for 3 times Set quota for 4 times Set quota for 5 times Set quota for 6 times Set quota for 7 times Set quota for 8 times (dd_pid=123782, time=0)successful (dd_pid=123784, time=0)successful Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 21 (97s) == sanity-quota test 22: enable/disable quota by 'lctl conf_param/set_param -P' ========================================================== 10:11:01 (1787753461) Set both mdt & ost quota type as ug Restart... Stopping clients: oleg118-client.virtnet /mnt/lustre (opts:) Stopping client oleg118-client.virtnet /mnt/lustre opts: Stopping clients: oleg118-client.virtnet /mnt/lustre2 (opts:) Stopping /mnt/lustre-mds1 (opts:-f) on oleg118-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg118-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg118-server sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime) proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755) securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev) devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000) tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755) tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755) cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd) pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime) bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event) cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids) cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,rdma) cgroup on /sys/fs/cgroup/misc type cgroup (rw,nosuid,nodev,noexec,relatime,misc) none on /sys/kernel/tracing type tracefs (rw,relatime) configfs on /sys/kernel/config type configfs (rw,relatime) /dev/nbd0 on / type ext4 (ro,relatime,stripe=32) rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M) debugfs on /sys/kernel/debug type debugfs (rw,relatime) mqueue on /dev/mqueue type mqueue (rw,relatime) none on /mnt type ramfs (rw,relatime) /dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime) none on /var/lib/stateless/writable type tmpfs (rw,relatime) none on /var/cache/man type tmpfs (rw,relatime) none on /var/log type tmpfs (rw,relatime) none on /var/lib/dbus type tmpfs (rw,relatime) none on /tmp type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) none on /var/tmp type tmpfs (rw,relatime) none on /var/lib/NetworkManager type tmpfs (rw,relatime) none on /var/lib/systemd/random-seed type tmpfs (rw,relatime) none on /var/spool type tmpfs (rw,relatime) none on /var/lib/nfs type tmpfs (rw,relatime) none on /var/lib/gssproxy type tmpfs (rw,relatime) none on /var/lib/logrotate type tmpfs (rw,relatime) none on /etc type tmpfs (rw,relatime) none on /var/lib/rsyslog type tmpfs (rw,relatime) none on /root/.ssh type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) 192.168.200.253:/exports/state/oleg118-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.18,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg118-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.18,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg118-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.18,local_lock=none,addr=192.168.200.253) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=378180k,mode=700) 192.168.200.253:/exports/testreports/68743/testresults/sanity-quota-zfs-rocky8.10_x86_64-rocky8.10_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.18,local_lock=none,addr=192.168.200.253) /dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime) Checking servers environments Checking clients oleg118-client.virtnet environments /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg118-server' oleg118-server: oleg118-server.virtnet: executing load_modules_local oleg118-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg118-server: Loading modules from /home/green/git/lustre-release/lustre oleg118-server: detected 4 online CPUs by sysfs oleg118-server: MODOPTS_LIBCFS= oleg118-server: Force libcfs to create 2 CPU partitions Setup mgs, mdt, osts Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg118-server: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg118-client: oleg118-server: ssh exited with exit code 1 Started lustre-MDT0000 Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg118-server: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg118-client: oleg118-server: ssh exited with exit code 1 Started lustre-OST0000 Start ost2: mount -t lustre -o localrecov lustre-ost2/ost2 /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg118-server: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg118-client: oleg118-server: ssh exited with exit code 1 Started lustre-OST0001 Starting client: oleg118-client.virtnet: -o user_xattr,flock 192.168.201.118@tcp:/lustre /mnt/lustre Starting client oleg118-client.virtnet: -o user_xattr,flock 192.168.201.118@tcp:/lustre /mnt/lustre Started clients oleg118-client.virtnet: 192.168.201.118@tcp:/lustre on /mnt/lustre type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8ffb491ad000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8ffb491ad000.idle_timeout=debug Verify if quota is enabled Set both mdt & ost quota type as none Restart... Stopping clients: oleg118-client.virtnet /mnt/lustre (opts:) Stopping client oleg118-client.virtnet /mnt/lustre opts: Stopping clients: oleg118-client.virtnet /mnt/lustre2 (opts:) Stopping /mnt/lustre-mds1 (opts:-f) on oleg118-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg118-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg118-server sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime) proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755) securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev) devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000) tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755) tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755) cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd) pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime) bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event) cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids) cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,rdma) cgroup on /sys/fs/cgroup/misc type cgroup (rw,nosuid,nodev,noexec,relatime,misc) none on /sys/kernel/tracing type tracefs (rw,relatime) configfs on /sys/kernel/config type configfs (rw,relatime) /dev/nbd0 on / type ext4 (ro,relatime,stripe=32) rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M) debugfs on /sys/kernel/debug type debugfs (rw,relatime) mqueue on /dev/mqueue type mqueue (rw,relatime) none on /mnt type ramfs (rw,relatime) /dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime) none on /var/lib/stateless/writable type tmpfs (rw,relatime) none on /var/cache/man type tmpfs (rw,relatime) none on /var/log type tmpfs (rw,relatime) none on /var/lib/dbus type tmpfs (rw,relatime) none on /tmp type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) none on /var/tmp type tmpfs (rw,relatime) none on /var/lib/NetworkManager type tmpfs (rw,relatime) none on /var/lib/systemd/random-seed type tmpfs (rw,relatime) none on /var/spool type tmpfs (rw,relatime) none on /var/lib/nfs type tmpfs (rw,relatime) none on /var/lib/gssproxy type tmpfs (rw,relatime) none on /var/lib/logrotate type tmpfs (rw,relatime) none on /etc type tmpfs (rw,relatime) none on /var/lib/rsyslog type tmpfs (rw,relatime) none on /root/.ssh type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) 192.168.200.253:/exports/state/oleg118-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.18,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg118-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.18,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg118-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.18,local_lock=none,addr=192.168.200.253) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=378180k,mode=700) 192.168.200.253:/exports/testreports/68743/testresults/sanity-quota-zfs-rocky8.10_x86_64-rocky8.10_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.18,local_lock=none,addr=192.168.200.253) /dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime) Checking servers environments Checking clients oleg118-client.virtnet environments /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg118-server' oleg118-server: oleg118-server.virtnet: executing load_modules_local oleg118-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg118-server: Loading modules from /home/green/git/lustre-release/lustre oleg118-server: detected 4 online CPUs by sysfs oleg118-server: MODOPTS_LIBCFS= oleg118-server: Force libcfs to create 2 CPU partitions Setup mgs, mdt, osts Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg118-server: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg118-client: oleg118-server: ssh exited with exit code 1 Started lustre-MDT0000 Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg118-server: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg118-client: oleg118-server: ssh exited with exit code 1 Started lustre-OST0000 Start ost2: mount -t lustre -o localrecov lustre-ost2/ost2 /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg118-server: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg118-client: oleg118-server: ssh exited with exit code 1 Started lustre-OST0001 Starting client: oleg118-client.virtnet: -o user_xattr,flock 192.168.201.118@tcp:/lustre /mnt/lustre Starting client oleg118-client.virtnet: -o user_xattr,flock 192.168.201.118@tcp:/lustre /mnt/lustre Started clients oleg118-client.virtnet: 192.168.201.118@tcp:/lustre on /mnt/lustre type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8ffb4a00f000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8ffb4a00f000.idle_timeout=debug Verify if quota is disabled PASS 22 (316s) == sanity-quota test 23: Quota should be honored with directIO (b16125) ========================================================== 10:16:17 (1787753777) SKIP: sanity-quota test_23 Overwrite in place is not guaranteed to be space neutral on ZFS SKIP 23 (7s) == sanity-quota test 24: lfs draws an asterix when limit is reached (b16646) ========================================================== 10:16:24 (1787753784) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Set user quota (limit: 5M) running as uid/gid/euid/egid 0/0/0/0, groups: 0 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d24.sanity-quota/f24.sanity-quota] [count=6] 6+0 records in 6+0 records out 6291456 bytes (6.3 MB, 6.0 MiB) copied, 1.07435 s, 5.9 MB/s /mnt/lustre 6149* 0 5120 - 1* 0 0 - lustre-MDT0000 2* - 2 - 1* - 0 - lustre-OST0000 6148* - 6148 - 1* - 0 - Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 24 (76s) == sanity-quota test 25: check indexes versions ========== 10:17:40 (1787753860) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Creating new pool qpool1 oleg118-server: Pool lustre.qpool1 created Adding targets to pool oleg118-server: OST lustre-OST0000_UUID added to pool lustre.qpool1 oleg118-server: OST lustre-OST0001_UUID added to pool lustre.qpool1 -u quota_usr space:0 Write... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d25.sanity-quota/f25.sanity-quota-0] [count=5] 5+0 records in 5+0 records out 5242880 bytes (5.2 MB, 5.0 MiB) copied, 1.48163 s, 3.5 MB/s Write out of block quota ... running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d25.sanity-quota/f25.sanity-quota-0] [count=5] [seek=5] dd: error writing '/mnt/lustre/d25.sanity-quota/f25.sanity-quota-0': Disk quota exceeded 5+0 records in 4+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 1.55674 s, 2.7 MB/s running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d25.sanity-quota/f25.sanity-quota-0] [count=1] [seek=10] dd: error writing '/mnt/lustre/d25.sanity-quota/f25.sanity-quota-0': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.27006 s, 0.0 kB/s Destroy the created pools: qpool1 lustre.qpool1 oleg118-server: OST lustre-OST0000_UUID removed from pool lustre.qpool1 oleg118-server: OST lustre-OST0001_UUID removed from pool lustre.qpool1 oleg118-server: Pool lustre.qpool1 destroyed Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 25 (132s) == sanity-quota test 27a: lfs quota/setquota should handle wrong arguments (b19612) ========================================================== 10:19:52 (1787753992) Disk quotas for usr root (uid 0): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace lfs quota: '43/0' dev not on a mounted Lustre filesystem Disk quotas for grp root (gid 0): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace lfs quota: '43/0' dev not on a mounted Lustre filesystem lfs setquota: either -u or -g must be specified setquota failed: Unknown error -4 Set filesystem quotas. usage: setquota [-t] {-u|-U|-g|-G|-p|-P ID} {-b|-B|-i|-I LIMIT} [--lqa LQA_NAME] [--pool POOL] MOUNT_POINT setquota {-u|-g|-p ID} {--default|--delete} MOUNT_POINT PASS 27a (8s) == sanity-quota test 27b: lfs quota/setquota should handle user/group/project ID (b20200) ========================================================== 10:20:00 (1787754000) lfs setquota: warning: block softlimit '1000' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details lfs setquota: warning: block hardlimit '1000' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details lfs setquota: warning: inode softlimit '1000' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details lfs setquota: warning: inode hardlimit '1000' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details lfs setquota: warning: block softlimit '1000' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details lfs setquota: warning: block hardlimit '1000' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details lfs setquota: warning: inode softlimit '1000' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details lfs setquota: warning: inode hardlimit '1000' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details lfs setquota: warning: block softlimit '1000' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details lfs setquota: warning: block hardlimit '1000' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details lfs setquota: warning: inode softlimit '1000' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details lfs setquota: warning: inode hardlimit '1000' smaller than minimum qunit size See 'lfs help setquota' or Lustre manual for details Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 1000 1000 - 0 1000 1000 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for grp quota_usr (gid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 1000 1000 - 0 1000 1000 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 Disk quotas for prj (pid 1000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 0 1000 1000 - 0 1000 1000 - lustre-MDT0000 0 - 0 - 0 - 0 - lustre-OST0000 0 - 0 - 0 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 0 PASS 27b (12s) == sanity-quota test 27c: lfs quota should support human-readable output ========================================================== 10:20:12 (1787754012) PASS 27c (9s) == sanity-quota test 27d: lfs setquota should support fraction block limit ========================================================== 10:20:21 (1787754021) PASS 27d (8s) == sanity-quota test 30: Hard limit updates should not reset grace times ========================================================== 10:20:29 (1787754029) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d30.sanity-quota/f30.sanity-quota] [count=8] 8+0 records in 8+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 2.05831 s, 4.1 MB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 8197* 4096 0 expired 1* 0 0 - lustre-MDT0000 2* - 2 - 1* - 0 - lustre-OST0000 8196 - 9220 - 1 - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 9220 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d30.sanity-quota/f30.sanity-quota] [conv=notrunc] [oflag=append] [count=4] dd: error writing '/mnt/lustre/d30.sanity-quota/f30.sanity-quota': Disk quota exceeded 2+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.499644 s, 2.1 MB/s Disk quotas for usr quota_usr (uid 60000): Filesystem kbytes bquota blimit bgrace files iquota ilimit igrace /mnt/lustre 9221* 4096 0 expired 1* 0 0 - lustre-MDT0000 2* - 2 - 1* - 0 - lustre-OST0000 9220* - 9220 - 1* - 0 - lustre-OST0001 0 - 0 - 0 - 0 - Total allocated inode limit: 0, total allocated block limit: 9220 running as uid/gid/euid/egid 60000/60000/60000/60000, groups: 60000 [dd] [if=/dev/urandom] [bs=1M] [of=/mnt/lustre/d30.sanity-quota/f30.sanity-quota] [conv=notrunc] [oflag=append] [count=4] dd: error writing '/mnt/lustre/d30.sanity-quota/f30.sanity-quota': Disk quota exceeded 1+0 records in 0+0 records out 0 bytes copied, 0.146151 s, 0.0 kB/s Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 30 (66s) == sanity-quota test 33: Basic usage tracking for user & group & project ========================================================== 10:21:35 (1787754095) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u 60000 space:0 -g 60000 space:0 -p 1000 space:0 Write files... /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d33.sanity-quota/f33.sanity-quota-0 Iteration 0/10 completed /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d33.sanity-quota/f33.sanity-quota-1 Iteration 1/10 completed /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d33.sanity-quota/f33.sanity-quota-2 Iteration 2/10 completed /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d33.sanity-quota/f33.sanity-quota-3 Iteration 3/10 completed /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d33.sanity-quota/f33.sanity-quota-4 Iteration 4/10 completed /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d33.sanity-quota/f33.sanity-quota-5 Iteration 5/10 completed /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d33.sanity-quota/f33.sanity-quota-6 Iteration 6/10 completed /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d33.sanity-quota/f33.sanity-quota-7 Iteration 7/10 completed /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d33.sanity-quota/f33.sanity-quota-8 Iteration 8/10 completed /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d33.sanity-quota/f33.sanity-quota-9 Iteration 9/10 completed /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d33.sanity-quota/f33.sanity-quota-10 Iteration 10/10 completed Wait for setattr on objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Verify disk usage after write -u 60000 space:22572 -g 60000 space:22572 -p 1000 space:22572 Verify inode usage after write -u 60000 inodes:11 -g 60000 inodes:11 -p 1000 inodes:11 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Verify disk usage after delete -u 60000 space:0 -u 60000 inodes:0 -g 60000 space:0 -g 60000 inodes:0 -p 1000 space:0 -p 1000 inodes:0 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS Waiting for MDT destroys to complete PASS 33 (176s) == sanity-quota test 34: Usage transfer for user & group & project ========================================================== 10:24:31 (1787754271) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u 60000 space:0 -g 60000 space:0 -u 60001 space:0 -p 1000 space:0 Write file... chown the file to user 60000 Wait for setattr on objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Verify disk usage for user 60000 -u 60000 space:2052 -u 60000 inodes:1 chgrp the file to group 60000 Wait for setattr on objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Verify disk usage for group 60000 -g 60000 space:2052 -g 60000 inodes:1 chown the file to user 60001 Wait for setattr on objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete change_project project id to 1000 /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d34.sanity-quota/f34.sanity-quota Wait for setattr on objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Verify disk usage for user 60001/60000 and group 60000 -u 60001 space:2052 -u 60000 space:0 -g 60000 space:2052 -p 1000 space:2052 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 34 (208s) == sanity-quota test 35: Usage is still accessible across reboot ========================================================== 10:27:59 (1787754479) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 Write file... /home/green/git/lustre-release/lustre/utils/lfs project -p 1000 /mnt/lustre/d35.sanity-quota/f35.sanity-quota Wait for setattr on objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete Save disk usage before restart -u 60000 space:2052 -u 60000 inodes:1 User 60000: 2052KB 1 inodes -g 60000 space:2052 -g 60000 inodes:1 Group 60000: 2052KB 1 inodes -p 1000 space:2052 -p 1000 inodes:1 Project 1000: 2052KB 1 inodes Restart... Stopping clients: oleg118-client.virtnet /mnt/lustre (opts:) Stopping client oleg118-client.virtnet /mnt/lustre opts: Stopping clients: oleg118-client.virtnet /mnt/lustre2 (opts:) Stopping /mnt/lustre-mds1 (opts:-f) on oleg118-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg118-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg118-server Checking servers environments Checking clients oleg118-client.virtnet environments /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg118-server' oleg118-server: oleg118-server.virtnet: executing load_modules_local oleg118-server: Loading modules from /home/green/git/lustre-release/lustre oleg118-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg118-server: detected 4 online CPUs by sysfs oleg118-server: MODOPTS_LIBCFS= oleg118-server: Force libcfs to create 2 CPU partitions Setup mgs, mdt, osts Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg118-server: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg118-client: oleg118-server: ssh exited with exit code 1 Started lustre-MDT0000 Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg118-server: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg118-client: oleg118-server: ssh exited with exit code 1 Started lustre-OST0000 Start ost2: mount -t lustre -o localrecov lustre-ost2/ost2 /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg118-server: oleg118-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg118-client: oleg118-server: ssh exited with exit code 1 Started lustre-OST0001 Starting client: oleg118-client.virtnet: -o user_xattr,flock 192.168.201.118@tcp:/lustre /mnt/lustre Starting client oleg118-client.virtnet: -o user_xattr,flock 192.168.201.118@tcp:/lustre /mnt/lustre Started clients oleg118-client.virtnet: 192.168.201.118@tcp:/lustre on /mnt/lustre type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8ffb481c8000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8ffb481c8000.idle_timeout=debug affected facets: Verify disk usage after restart -u 60000 space:2052 -u 60000 inodes:1 -g 60000 space:2052 -g 60000 inodes:1 Append to the same file... Verify space usage is increased -u 60000 space:3076 -g 60000 space:3076 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 35 (271s) == sanity-quota test 37: Quota accounted properly for file created by 'lfs setstripe' ========================================================== 10:32:30 (1787754750) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u 60000 space:0 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.474285 s, 2.2 MB/s sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete -u 60000 space:1027 Delete files... Wait for unlink objects finished... sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 37 (85s) == sanity-quota test 38: Quota accounting iterator doesn't skip id entries ========================================================== 10:33:55 (1787754835) sleep 5 for ZFS MDS Waiting for MDT destroys to complete Creating test directory fail_val=0 fail_loc=0 -u 60000 space:0 -u 60001 space:0 Create 10000 files...