-----============= acceptance-small: sanityn ============----- Mon May 25 04:27:18 EDT 2026 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg110-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 40a 102 skipping tests SLOW=no: 33a === sanityn: start setup 04:27:49 (1779697669) === Starting client oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre2 Mount client oleg110-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre2 Started clients oleg110-client.virtnet: 192.168.201.110@tcp:/lustre on /mnt/lustre2 type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) oleg110-client.virtnet: executing check_config_client /mnt/lustre oleg110-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg110-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff9ae5833e3800.idle_timeout=debug osc.lustre-OST0000-osc-ffff9ae584d36800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9ae5833e3800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9ae584d36800.idle_timeout=debug disable quota as required oleg110-server: oleg110-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 04:28:33 (1779697713) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0795152 s, 13.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7601] == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 04:28:37 (1779697717) PASS 0 (10s) == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 04:28:47 (1779697727) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (9s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 04:28:56 (1779697736) -rw-r--r-- 1 root root 0 May 25 04:28 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (9s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 04:29:05 (1779697745) -rw-r--r-- 1 root root 0 May 25 04:29 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (9s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 04:29:14 (1779697754) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (9s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 04:29:23 (1779697763) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (8s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 04:29:31 (1779697771) total 1 -rwxrwxrwx 1 root root 0 May 25 04:28 f2a -rwxrwxrwx 1 root root 0 May 25 04:29 f2b total 1 -rwxrwxrwx 1 root root 0 May 25 04:28 f2a -rwxrwxrwx 1 root root 0 May 25 04:29 f2b running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.0145616 s, 35.2 kB/s PASS 2e (8s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 04:29:40 (1779697780) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (4s) == sanityn test 2g: check blocks update on sync write ==== 04:29:44 (1779697784) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.520407 s, 4.0 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (9s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 04:29:53 (1779697793) PASS 3 (7s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 04:30:00 (1779697800) PASS 4 (9s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 04:30:09 (1779697809) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (10s) == sanityn test 6: remove of open file on other node ============================================================================ 04:30:19 (1779697819) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (8s) == sanityn test 7: remove of open directory on other node ======================================================================= 04:30:27 (1779697827) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (8s) == sanityn test 8: remove of open special file on other node ==================================================================== 04:30:36 (1779697836) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (10s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 04:30:46 (1779697846) PASS 9a (10s) == sanityn test 9b: append to striped sparse file ======== 04:30:55 (1779697855) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0722001 s, 14.5 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.0835765 s, 0.0 kB/s Data read (expecting 'foo'): foo PASS 9b (9s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 04:31:05 (1779697865) 1+0 records in 1+0 records out 1 byte copied, 0.0236193 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0219277 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0157654 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0150795 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0145573 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00839941 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0147294 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0194735 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0266419 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0193863 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0229229 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0119659 s, 0.1 kB/s PASS 10a (10s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 04:31:15 (1779697875) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.0176507 s, 174 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0464256 s, 88.2 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.00197887 s, 1.6 MB/s PASS 10b (9s) == sanityn test 11: execution of file opened for write should return error ============================================================== 04:31:24 (1779697884) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 300: /mnt/lustre2/d11/f: Text file busy PASS 11 (9s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 04:31:34 (1779697894) warning: '-runtime' deprecated, use '-t runtime' instead - link 238 (time 1779697907.43 total 10.06 last 23.65) - link 463 (time 1779697917.48 total 20.11 last 22.40) - link 693 (time 1779697927.49 total 30.12 last 22.97) - link 942 (time 1779697937.51 total 40.14 last 24.86) - link 1171 (time 1779697947.56 total 50.19 last 22.78) - link 1392 (time 1779697957.59 total 60.22 last 22.04) - link 1603 (time 1779697967.60 total 70.23 last 21.07) - link 1841 (time 1779697977.62 total 80.25 last 23.75) - link 2085 (time 1779697987.65 total 90.28 last 24.33) total: 2337 link in 100.04 seconds: 23.36 ops/second using seed 3823468365 running for 100 seconds - stat 3033 (time 1779697909 ; total 11 ; last 11) - stat 6234 (time 1779697920 ; total 22 ; last 11) - stat 9736 (time 1779697931 ; total 33 ; last 11) - stat 10000 (time 1779697931 ; total 33 ; last 0) - stat 13199 (time 1779697942 ; total 44 ; last 11) - stat 15851 (time 1779697953 ; total 55 ; last 11) - stat 17437 (time 1779697964 ; total 66 ; last 11) - stat 18969 (time 1779697975 ; total 77 ; last 11) - stat 20000 (time 1779697982 ; total 84 ; last 7) - stat 21658 (time 1779697993 ; total 95 ; last 11) total: 22649 stats in 100 seconds: 226.490005 stats/second - unlinked 0 (time 1779697948 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile2337) error: No such file or directory total: 2336 unlinks in 85 seconds: 27.482353 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (21402) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (21404) - No such process PASS 12 (148s) == sanityn test 13: test directory page revocation ======= 04:34:01 (1779698041) f13 f13 PASS 13 (13s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 04:34:14 (1779698054) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 5002: 22590 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (9s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 04:34:23 (1779698063) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 5002: 23169 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (10s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 04:34:33 (1779698073) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 350: 23748 Terminated $DIR1/$tdir/sleep 60 PASS 14b (9s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 04:34:42 (1779698082) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 362: 24331 Terminated $DIR1/$tdir/sleep 60 PASS 14c (11s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 04:34:53 (1779698093) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 373: 24914 Terminated $DIR1/$tdir/sleep 60 PASS 14d (11s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 04:35:05 (1779698105) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky sleep 5 for ZFS MDS Waiting for MDT destroys to complete fallocate on zfs doesn't consume space STRIPECOUNT=2 ORIGFREE=7529472 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7529472KB free > 800000KB, increase MAXFREE (or reduce fs size) sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (65s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 04:36:10 (1779698170) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.438719 s, 19.1 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.430348 s, 19.5 MB/s Chance of close/open is 1 in 50 Seed set to 8174 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x9d7e70 000007[0] 1779698176.247756 trunc from 00000000 to 0x9d7e6f (0x9d7e70 bytes) 000009[0] 1779698176.478340 write 0x2ea40c thru 0x2f7f6a (0xdb5f bytes) 000015[1] 1779698176.761226 write 0x8928bc thru 0x89c3f2 (0x9b37 bytes) 000016[1] 1779698176.871546 mapread 0x58f37b thru 0x5930f9 (0x3d7f bytes) 000017[0] 1779698177.008906 mapread 0x0bab8f thru 0x0c79f5 (0xce67 bytes) 000018[0] 1779698177.180578 read 0x720d1b thru 0x7280d7 (0x73bd bytes) 000019[0] 1779698177.200191 mapread 0x515a63 thru 0x523cea (0xe288 bytes) 000021[1] 1779698177.263129 mapread 0x7c4e0f thru 0x7cb36f (0x6561 bytes) 000023[0] 1779698177.298925 mapread 0x84435c thru 0x852878 (0xe51d bytes) 000028[0] 1779698177.382302 read 0x2ba293 thru 0x2bca7e (0x27ec bytes) 000029[1] 1779698177.396080 mapwrite 0x08a35a thru 0x097d95 (0xda3c bytes) 000030[1] 1779698177.649420 trunc from 0x9d7e70 to 0x605129 (0x3d2d46 bytes) 000032[0] 1779698177.835219 mapwrite 0x2aca09 thru 0x2b84e6 (0xbade bytes) 000033[1] 1779698178.000296 trunc from 0x60512a to 0x108817 (0x4fc912 bytes) 000037[0] 1779698178.131334 write 0x25d695 thru 0x26abf6 (0xd562 bytes) 000038[0] 1779698178.177467 mapread 0x1c7fd4 thru 0x1d3f69 (0xbf96 bytes) 000039[0] 1779698178.235221 trunc from 0x26abf7 to 0x093b3e (0x1d70b8 bytes) 000040[0] 1779698178.280933 mapwrite 0x232973 thru 0x23dd19 (0xb3a7 bytes) 000041[1] 1779698178.404420 mapwrite 0x77aef8 thru 0x77bfbf (0x10c8 bytes) 000042[1] 1779698178.589144 write 0x057f9f thru 0x0624b3 (0xa515 bytes) 000044[1] 1779698178.668911 trunc from 0x77bfc0 to 0x7ec3e6 (0x70427 bytes) 000045[1] 1779698178.755131 read 0x3172a2 thru 0x31c758 (0x54b7 bytes) 000046[0] 1779698178.778135 read 0x458954 thru 0x45b740 (0x2ded bytes) 000047[0] 1779698178.819745 mapread 0x045e28 thru 0x04e66f (0x8848 bytes) 000048[0] 1779698178.993378 mapread 0x302fdc thru 0x310ba4 (0xdbc9 bytes) 000049[0] 1779698179.037562 read 0x705636 thru 0x7067a8 (0x1173 bytes) 000050[1] 1779698179.064697 mapread 0x1a0c7b thru 0x1a33c4 (0x274a bytes) 000054[0] 1779698179.146236 mapread 0x7274e2 thru 0x73260e (0xb12d bytes) 000055[1] 1779698179.272797 mapread 0x74b081 thru 0x74cb08 (0x1a88 bytes) 000056[1] 1779698179.292019 mapwrite 0x75333f thru 0x75d93d (0xa5ff bytes) 000058[0] 1779698179.592715 mapread 0x3338e8 thru 0x33dba5 (0xa2be bytes) 000059[1] 1779698179.711726 mapwrite 0x2c1e28 thru 0x2c6004 (0x41dd bytes) 000060[1] 1779698179.879658 mapwrite 0x1eefe9 thru 0x1f29f4 (0x3a0c bytes) 000061[1] 1779698180.001554 mapwrite 0x4eae56 thru 0x4efd5d (0x4f08 bytes) 000062[1] 1779698180.130504 write 0x5720b0 thru 0x57e653 (0xc5a4 bytes) 000063[1] 1779698180.204601 mapwrite 0x8bb909 thru 0x8c34ca (0x7bc2 bytes) 000064[0] 1779698180.375147 mapread 0x7c9d63 thru 0x7d8025 (0xe2c3 bytes) 000069[1] 1779698180.579507 read 0x08b6b5 thru 0x094444 (0x8d90 bytes) 000071[1] 1779698180.626446 write 0x5134ae thru 0x51febd (0xca10 bytes) 000072[1] 1779698180.663348 trunc from 0x8c34cb to 0x7c99e0 (0xf9aea bytes) 000073[0] 1779698180.734561 mapwrite 0x3fc1ee thru 0x4042b3 (0x80c6 bytes) 000074[0] 1779698181.004176 mapread 0x080a83 thru 0x08a588 (0x9b06 bytes) 000075[0] 1779698181.044391 mapread 0x26685e thru 0x267f20 (0x16c3 bytes) 000076[0] 1779698181.131394 mapread 0x57e707 thru 0x588c99 (0xa593 bytes) 000079[1] 1779698181.296258 write 0x4f6b2c thru 0x4fbd92 (0x5267 bytes) 000080[1] 1779698181.399497 trunc from 0x7c99e1 to 0x4d62ec (0x2f36f4 bytes) 000082[0] 1779698181.604037 read 0x05c6af thru 0x0640c4 (0x7a16 bytes) 000083[1] 1779698181.684198 mapread 0x4b8bf4 thru 0x4bcab2 (0x3ebf bytes) 000085[0] 1779698181.725425 write 0x5ae26f thru 0x5b0527 (0x22b9 bytes) 000086[0] 1779698181.838701 mapwrite 0x87e5a3 thru 0x8889cc (0xa42a bytes) 000088[0] 1779698182.183273 trunc from 0x8889cd to 0x3336cb (0x555301 bytes) 000090[1] 1779698182.255189 mapread 0x006f6d thru 0x008ad4 (0x1b68 bytes) 000093[1] 1779698182.389644 write 0x21be6c thru 0x21d41d (0x15b2 bytes) 000095[0] 1779698182.454222 mapwrite 0x6e024c thru 0x6eca94 (0xc849 bytes) 000096[1] 1779698182.800815 mapread 0x24035c thru 0x24fd97 (0xfa3c bytes) 000098[1] 1779698183.011064 write 0x72f147 thru 0x732a6f (0x3929 bytes) 000100[0] 1779698183.141675 write 0x7a291a thru 0x7a51c8 (0x28af bytes) 000101[0] 1779698183.246398 read 0x57df23 thru 0x58aebd (0xcf9b bytes) 000102[0] 1779698183.301270 read 0x669f8b thru 0x6790f2 (0xf168 bytes) 000103[1] 1779698183.324206 mapwrite 0x9514d7 thru 0x95c57f (0xb0a9 bytes) 000106[1] 1779698183.542827 mapwrite 0x3eadb1 thru 0x3f1a95 (0x6ce5 bytes) 000109[1] 1779698183.861245 trunc from 0x95c580 to 0x89e4cc (0xbe0b3 bytes) 000110[1] 1779698184.016550 write 0x138d5a thru 0x14817b (0xf422 bytes) 000114[0] 1779698184.203305 mapwrite 0x4fbbdf thru 0x504b3a (0x8f5c bytes) 000115[0] 1779698184.397922 mapwrite 0x15a795 thru 0x160482 (0x5cee bytes) 000117[1] 1779698184.546902 read 0x23a8cb thru 0x243060 (0x8796 bytes) 000118[1] 1779698184.615167 trunc from 0x89e4cd to 0x18d33c (0x711190 bytes) 000120[1] 1779698184.699063 read 0x0944f4 thru 0x0a2a9c (0xe5a9 bytes) 000130[1] 1779698185.039496 mapwrite 0x96f36b thru 0x9782b5 (0x8f4b bytes) 000132[0] 1779698185.375760 trunc from 0x9782b6 to 0x038819 (0x93fa9c bytes) 000133[1] 1779698185.446135 mapwrite 0x931bec thru 0x93c806 (0xac1b bytes) 000135[1] 1779698185.676876 read 0x5c92d9 thru 0x5cf71a (0x6442 bytes) 000136[1] 1779698185.690309 mapread 0x4e0121 thru 0x4e82ff (0x81df bytes) 000137[0] 1779698185.715112 mapwrite 0x433e57 thru 0x438a95 (0x4c3f bytes) 000138[1] 1779698185.969539 read 0x6e8177 thru 0x6f5dc4 (0xdc4e bytes) 000139[1] 1779698186.089699 mapwrite 0x2ea939 thru 0x2f0045 (0x570d bytes) 000141[1] 1779698186.393286 mapwrite 0x845954 thru 0x84f025 (0x96d2 bytes) 000145[1] 1779698186.630751 mapread 0x62bbb9 thru 0x6355ad (0x99f5 bytes) 000147[0] 1779698186.681024 trunc from 0x93c807 to 0x825d3d (0x116ac9 bytes) 000148[1] 1779698186.757143 read 0x62022f thru 0x625918 (0x56ea bytes) 000154[0] 1779698186.860562 write 0x60033b thru 0x6092a7 (0x8f6d bytes) 000156[1] 1779698186.932257 read 0x00946d thru 0x00d2a0 (0x3e34 bytes) 000157[0] 1779698186.963305 trunc from 0x825d3e to 0x6be0a6 (0x167c97 bytes) 000160[0] 1779698187.091771 trunc from 0x6be0a7 to 0x46757f (0x256b27 bytes) 000161[0] 1779698187.123183 trunc from 0x467580 to 0x290005 (0x1d757a bytes) 000162[0] 1779698187.142027 mapread 0x065a3f thru 0x072a21 (0xcfe3 bytes) 000164[1] 1779698187.231777 trunc from 0x290006 to 0x8db1a3 (0x64b19e bytes) 000165[0] 1779698187.318506 write 0x213c58 thru 0x2176f8 (0x3aa1 bytes) 000167[0] 1779698187.408232 write 0x42741d thru 0x427e5e (0xa42 bytes) 000168[0] 1779698187.437416 trunc from 0x8db1a4 to 0x6ff531 (0x1dbc72 bytes) 000170[0] 1779698187.552238 trunc from 0x6ff532 to 0x690a64 (0x6eacd bytes) 000172[0] 1779698187.583735 read 0x24a9ce thru 0x25a276 (0xf8a9 bytes) 000173[1] 1779698187.600540 trunc from 0x690a65 to 0x1f9c98 (0x496dcc bytes) 000174[0] 1779698187.709140 read 0x0b710f thru 0x0b7326 (0x218 bytes) 000175[1] 1779698187.813630 write 0x8e579d thru 0x8f2412 (0xcc76 bytes) 000178[1] 1779698187.924480 mapread 0x5895cb thru 0x589ef2 (0x928 bytes) 000179[0] 1779698188.018842 mapwrite 0x7a66c3 thru 0x7ab7f4 (0x5132 bytes) 000180[0] 1779698188.434048 trunc from 0x8f2413 to 0x0e0447 (0x811fcb bytes) 000181[1] 1779698188.497406 mapread 0x09dd65 thru 0x0a83c3 (0xa65f bytes) 000189[0] 1779698188.721729 mapwrite 0x660ce6 thru 0x66f7d5 (0xeaf0 bytes) 000192[1] 1779698188.991011 mapread 0x28e92e thru 0x29c4a3 (0xdb76 bytes) 000193[1] 1779698189.076505 write 0x7b1b60 thru 0x7b7bed (0x608e bytes) 000195[1] 1779698189.096805 mapwrite 0x86b88d thru 0x871da3 (0x6517 bytes) 000196[1] 1779698189.310495 write 0x03a0a0 thru 0x04509a (0xaffb bytes) 000199[1] 1779698189.428273 read 0x74d0e9 thru 0x75c45f (0xf377 bytes) 000200[0] 1779698189.446025 mapwrite 0x8b5b47 thru 0x8c325d (0xd717 bytes) 000202[1] 1779698189.811315 write 0x35701b thru 0x35afd6 (0x3fbc bytes) 000204[0] 1779698189.986152 mapread 0x02d0af thru 0x03456a (0x74bc bytes) 000205[0] 1779698190.084015 mapwrite 0x635231 thru 0x63c1ad (0x6f7d bytes) 000207[0] 1779698190.200937 mapread 0x05d41f thru 0x05f07b (0x1c5d bytes) 000211[1] 1779698190.266871 read 0x1eb13e thru 0x1f9768 (0xe62b bytes) 000212[1] 1779698190.279484 trunc from 0x8c325e to 0x22b2f6 (0x697f67 bytes) 000215[1] 1779698190.424327 read 0x1a457e thru 0x1a9b62 (0x55e5 bytes) 000216[1] 1779698190.486244 mapread 0x068c83 thru 0x06e6d0 (0x5a4e bytes) 000217[1] 1779698190.517141 write 0x1aa932 thru 0x1adce3 (0x33b2 bytes) 000219[1] 1779698190.593547 read 0x20f967 thru 0x217444 (0x7ade bytes) 000221[1] 1779698190.597886 mapwrite 0x1f313b thru 0x1fb779 (0x863f bytes) 000222[1] 1779698190.856109 read 0x1ec6f0 thru 0x1ef0dd (0x29ee bytes) 000224[1] 1779698190.913800 write 0x9f7772 thru 0x9fffff (0x888e bytes) 000225[0] 1779698190.920202 mapread 0x979c5a thru 0x983d23 (0xa0ca bytes) 000228[1] 1779698191.049240 write 0x4b139b thru 0x4b1b4c (0x7b2 bytes) 000229[1] 1779698191.180928 mapread 0x237dd5 thru 0x2411cd (0x93f9 bytes) 000231[1] 1779698191.206521 write 0x2dc6ee thru 0x2e360c (0x6f1f bytes) 000232[1] 1779698191.243975 write 0x7ac768 thru 0x7afa76 (0x330f bytes) 000233[1] 1779698191.291188 trunc from 0xa00000 to 0x442a13 (0x5bd5ec bytes) 000234[1] 1779698191.360924 mapread 0x301731 thru 0x30dd27 (0xc5f7 bytes) 000246[0] 1779698191.682368 mapwrite 0x8072e4 thru 0x8090d6 (0x1df3 bytes) 000248[1] 1779698191.896945 read 0x4f5d3f thru 0x4f7f04 (0x21c6 bytes) 000249[1] 1779698191.928046 trunc from 0x8090d7 to 0x2698b5 (0x59f821 bytes) 000251[1] 1779698191.997122 read 0x1d8f26 thru 0x1e854b (0xf626 bytes) 000253[1] 1779698192.082863 trunc from 0x2698b6 to 0x453a75 (0x1ea1c0 bytes) 000256[1] 1779698192.226347 write 0x6ed5d6 thru 0x6f8281 (0xacac bytes) 000257[1] 1779698192.241495 write 0x75e514 thru 0x76b8d2 (0xd3bf bytes) 000259[0] 1779698192.246063 mapwrite 0x8d3bc1 thru 0x8dbcef (0x812f bytes) 000260[0] 1779698192.431017 mapread 0x1417eb thru 0x1459a6 (0x41bc bytes) 000261[0] 1779698192.511408 mapwrite 0x1ce535 thru 0x1d47e4 (0x62b0 bytes) 000262[0] 1779698192.605182 read 0x078ee4 thru 0x082ca8 (0x9dc5 bytes) 000263[0] 1779698192.621043 trunc from 0x8dbcf0 to 0x4e0a55 (0x3fb29a bytes) 000264[1] 1779698192.676001 read 0x11ab5e thru 0x11e7ab (0x3c4e bytes) 000265[0] 1779698192.719045 read 0x0d88ba thru 0x0e2b96 (0xa2dd bytes) 000268[0] 1779698192.774038 write 0x6236d2 thru 0x62cbc8 (0x94f7 bytes) 000271[1] 1779698192.801229 read 0x1c4a36 thru 0x1c5b36 (0x1101 bytes) 000273[0] 1779698192.825152 write 0x785db1 thru 0x794abe (0xed0e bytes) 000274[1] 1779698192.849173 mapwrite 0x4c85de thru 0x4cd5fc (0x501f bytes) 000275[1] 1779698192.988230 mapwrite 0x5f97ff thru 0x607572 (0xdd74 bytes) 000276[0] 1779698193.104258 mapread 0x5809bc thru 0x5881eb (0x7830 bytes) 000277[0] 1779698193.170011 mapread 0x40277e thru 0x40b88d (0x9110 bytes) 000280[0] 1779698193.209039 read 0x5cd58c thru 0x5d8607 (0xb07c bytes) 000281[1] 1779698193.230875 write 0x2454b1 thru 0x24cf59 (0x7aa9 bytes) 000282[0] 1779698193.275880 mapread 0x177ee2 thru 0x181644 (0x9763 bytes) 000286[0] 1779698193.360882 trunc from 0x794abf to 0x4efec6 (0x2a4bf8 bytes) 000288[1] 1779698193.470373 write 0x435a9c thru 0x4416fb (0xbc60 bytes) 000290[0] 1779698193.545847 write 0x371038 thru 0x3793e9 (0x83b2 bytes) 000296[0] 1779698193.705006 write 0x165c94 thru 0x172c84 (0xcff1 bytes) 000299[1] 1779698193.795259 read 0x0f7a95 thru 0x0fd0cc (0x5638 bytes) 000300[0] 1779698193.848165 read 0x4aea7c thru 0x4b3de9 (0x536e bytes) 000302[1] 1779698193.894734 write 0x9894d5 thru 0x98e8f2 (0x541e bytes) 000303[1] 1779698193.933666 mapwrite 0x75a224 thru 0x75c681 (0x245e bytes) 000306[1] 1779698194.099263 trunc from 0x98e8f3 to 0x641a25 (0x34cecd bytes) 000307[1] 1779698194.190737 read 0x3e7fa3 thru 0x3f5b7a (0xdbd8 bytes) 000309[0] 1779698194.263445 write 0x297c1d thru 0x2987c7 (0xbab bytes) 000311[1] 1779698194.352570 trunc from 0x641a26 to 0x65109b (0xf676 bytes) 000312[1] 1779698194.429533 mapread 0x36bae0 thru 0x3716d4 (0x5bf5 bytes) 000314[0] 1779698194.538178 mapwrite 0x7f0afa thru 0x7feb51 (0xe058 bytes) 000317[1] 1779698194.813413 mapread 0x763cd9 thru 0x77233e (0xe666 bytes) 000318[0] 1779698194.918238 read 0x6dbc0f thru 0x6dd650 (0x1a42 bytes) 000319[1] 1779698194.950651 trunc from 0x7feb52 to 0x62cab9 (0x1d2098 bytes) 000322[1] 1779698195.118368 trunc from 0x62caba to 0x44bc2b (0x1e0e8e bytes) 000323[0] 1779698195.155164 mapwrite 0x522fd8 thru 0x52b36b (0x8394 bytes) 000324[0] 1779698195.295084 trunc from 0x52b36c to 0x383990 (0x1a79db bytes) 000325[0] 1779698195.338763 mapread 0x247431 thru 0x24ca4e (0x561e bytes) 000326[0] 1779698195.389335 read 0x0f43c2 thru 0x0f8218 (0x3e57 bytes) 000333[1] 1779698195.817428 mapwrite 0x945d41 thru 0x955688 (0xf948 bytes) 000334[0] 1779698196.294742 mapwrite 0x73e2e5 thru 0x73f7d1 (0x14ed bytes) 000335[0] 1779698196.533756 trunc from 0x955689 to 0x098298 (0x8bd3f0 bytes) 000336[0] 1779698196.573058 write 0x4c7788 thru 0x4c8fae (0x1827 bytes) 000338[0] 1779698196.575574 write 0x3681ce thru 0x3732d4 (0xb107 bytes) 000340[0] 1779698196.642516 write 0x144f42 thru 0x14d4a6 (0x8565 bytes) 000342[0] 1779698196.716646 read 0x35f41f thru 0x3662df (0x6ec1 bytes) 000344[1] 1779698196.816244 mapread 0x01f97d thru 0x0262d8 (0x695c bytes) 000345[0] 1779698196.956921 mapread 0x4c54f9 thru 0x4c8fae (0x3ab6 bytes) 000346[0] 1779698196.998786 mapwrite 0x1105f6 thru 0x113348 (0x2d53 bytes) 000352[0] 1779698197.359678 mapread 0x14e6b7 thru 0x15cfe3 (0xe92d bytes) 000358[0] 1779698197.570773 trunc from 0x4c8faf to 0x051399 (0x477c15 bytes) 000362[1] 1779698197.749244 read 0x00c383 thru 0x011214 (0x4e92 bytes) 000365[1] 1779698197.799264 trunc from 0x05139a to 0x97fc13 (0x92e87a bytes) 000369[0] 1779698198.001551 trunc from 0x97fc14 to 0x7b8dba (0x1c6e59 bytes) 000370[1] 1779698198.103623 mapwrite 0x48a2fa thru 0x497448 (0xd14f bytes) 000371[0] 1779698198.416855 mapread 0x17ade1 thru 0x180bc9 (0x5de9 bytes) 000373[0] 1779698198.497142 mapwrite 0x7edbcf thru 0x7f7093 (0x94c5 bytes) 000374[1] 1779698198.706644 write 0x94ca0c thru 0x95ba6a (0xf05f bytes) 000377[1] 1779698198.789223 trunc from 0x95ba6b to 0x62ee0a (0x32cc60 bytes) 000378[0] 1779698198.859826 mapwrite 0x29c225 thru 0x2a6e96 (0xac72 bytes) 000380[0] 1779698199.029097 trunc from 0x62ee0b to 0x7029ef (0xd3be5 bytes) 000381[0] 1779698199.063115 trunc from 0x7029f0 to 0x81e2ff (0x11b910 bytes) 000386[1] 1779698199.124272 write 0x473d24 thru 0x4765a9 (0x2886 bytes) 000387[1] 1779698199.208332 mapwrite 0x5184ba thru 0x518ce2 (0x829 bytes) 000388[1] 1779698199.390392 write 0x3e9632 thru 0x3ef988 (0x6357 bytes) 000389[0] 1779698199.601356 mapwrite 0x4dbd60 thru 0x4ea4bf (0xe760 bytes) 000390[0] 1779698199.793304 mapread 0x45cc11 thru 0x4617e2 (0x4bd2 bytes) 000392[1] 1779698199.936579 read 0x273c64 thru 0x277ca9 (0x4046 bytes) 000394[0] 1779698199.986035 mapread 0x25c0c8 thru 0x25fdb8 (0x3cf1 bytes) 000397[0] 1779698200.164380 mapread 0x08e188 thru 0x099b5b (0xb9d4 bytes) 000400[0] 1779698200.296294 mapwrite 0x29221a thru 0x29fe22 (0xdc09 bytes) 000401[0] 1779698200.585141 mapwrite 0x19841f thru 0x1a1644 (0x9226 bytes) 000404[1] 1779698200.735401 mapread 0x06bc42 thru 0x075521 (0x98e0 bytes) 000406[0] 1779698200.818259 write 0x73cc11 thru 0x74429a (0x768a bytes) 000407[1] 1779698200.873394 read 0x4c345b thru 0x4c3c39 (0x7df bytes) 000409[1] 1779698200.922441 read 0x3c8fd5 thru 0x3d0acb (0x7af7 bytes) 000411[0] 1779698200.935459 mapwrite 0x187501 thru 0x18c87c (0x537c bytes) 000413[0] 1779698201.093504 mapwrite 0x89d9dd thru 0x8a3684 (0x5ca8 bytes) 000414[0] 1779698201.289832 write 0x74960f thru 0x75338d (0x9d7f bytes) 000415[1] 1779698201.344261 trunc from 0x8a3685 to 0x01d14c (0x886538 bytes) 000416[1] 1779698201.412773 mapread 0x01a64e thru 0x01d14c (0x2aff bytes) 000417[0] 1779698201.441699 trunc from 0x01d14d to 0x7985a6 (0x77b45a bytes) 000423[0] 1779698201.640076 write 0x820f1c thru 0x82a057 (0x913c bytes) 000424[1] 1779698201.653012 write 0x8053df thru 0x80d0c2 (0x7ce4 bytes) 000425[1] 1779698201.775422 mapread 0x2b26e7 thru 0x2bd1c4 (0xaade bytes) 000426[1] 1779698201.939454 mapwrite 0x1692c6 thru 0x16f2c9 (0x6004 bytes) 000427[1] 1779698202.169690 read 0x115675 thru 0x11db62 (0x84ee bytes) 000428[0] 1779698202.220936 mapwrite 0x96ed55 thru 0x97ab6c (0xbe18 bytes) 000430[0] 1779698202.571572 write 0x836752 thru 0x83d889 (0x7138 bytes) 000431[1] 1779698202.601411 write 0x2517de thru 0x252cb5 (0x14d8 bytes) 000433[0] 1779698202.712921 read 0x51d054 thru 0x51e74d (0x16fa bytes) 000434[1] 1779698202.760551 mapread 0x64cc71 thru 0x653726 (0x6ab6 bytes) 000438[1] 1779698202.937186 write 0x538a63 thru 0x53e39a (0x5938 bytes) 000439[1] 1779698203.038079 trunc from 0x97ab6d to 0x729cf6 (0x250e76 bytes) 000440[1] 1779698203.074895 mapread 0x6d3e74 thru 0x6e00b8 (0xc245 bytes) 000444[1] 1779698203.293480 read 0x3df9c3 thru 0x3ee3ac (0xe9ea bytes) 000446[1] 1779698203.346725 read 0x5f86d9 thru 0x606bbd (0xe4e5 bytes) 000448[0] 1779698203.400435 read 0x06f854 thru 0x07f7db (0xff88 bytes) 000450[0] 1779698203.504460 read 0x1a8502 thru 0x1ac7e3 (0x42e2 bytes) 000451[1] 1779698203.544451 write 0x0a3168 thru 0x0a8b45 (0x59de bytes) 000453[1] 1779698203.597366 write 0x86348c thru 0x869b76 (0x66eb bytes) 000456[0] 1779698203.637727 read 0x0afbc0 thru 0x0b492e (0x4d6f bytes) 000461[1] 1779698203.720339 read 0x19f558 thru 0x1a4073 (0x4b1c bytes) 000462[1] 1779698203.751138 read 0x36de38 thru 0x37dd53 (0xff1c bytes) 000464[0] 1779698203.770299 mapread 0x30b348 thru 0x31ace3 (0xf99c bytes) 000465[0] 1779698203.854148 mapwrite 0x6eb4cc thru 0x6f7767 (0xc29c bytes) 000469[0] 1779698204.092337 mapwrite 0x137088 thru 0x13ddfc (0x6d75 bytes) 000471[0] 1779698204.275075 read 0x1a17b6 thru 0x1a17e1 (0x2c bytes) 000474[0] 1779698204.382893 trunc from 0x869b77 to 0x41a16e (0x44fa08 bytes) 000477[0] 1779698204.495790 trunc from 0x41a16f to 0x23e843 (0x1db92b bytes) 000481[1] 1779698204.581323 write 0x424794 thru 0x42f221 (0xaa8e bytes) 000482[1] 1779698204.658196 mapread 0x01c775 thru 0x029d9a (0xd626 bytes) 000483[0] 1779698204.802534 mapread 0x330728 thru 0x33b150 (0xaa29 bytes) 000486[1] 1779698205.328336 trunc from 0x42f222 to 0x6a11de (0x271fbd bytes) 000487[0] 1779698205.418650 write 0x844d66 thru 0x847369 (0x2604 bytes) 000490[1] 1779698205.511545 trunc from 0x84736a to 0x304c34 (0x542735 bytes) 000491[0] 1779698205.580124 mapwrite 0x277f31 thru 0x2843d9 (0xc4a9 bytes) 000493[1] 1779698205.782969 trunc from 0x304c35 to 0x8a6e7a (0x5a2246 bytes) 000494[0] 1779698205.839932 mapread 0x2f9b16 thru 0x2fc776 (0x2c61 bytes) 000496[1] 1779698205.918719 trunc from 0x8a6e7b to 0x0ffede (0x7a6f9c bytes) 000498[0] 1779698206.001151 mapread 0x0a23bc thru 0x0abc2f (0x9874 bytes) 000499[1] 1779698206.093800 trunc from 0x0ffedf to 0x630593 (0x5306b5 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 8206 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[0] 1779698207.439348 write 0x1d1000 thru 0x1d757d (0x657e bytes) 000003[1] 1779698207.481673 read 0x002000 thru 0x00b199 (0x919a bytes) 000004[1] 1779698207.523127 mapread 0x112000 thru 0x11797e (0x597f bytes) 000006[1] 1779698207.574624 read 0x125000 thru 0x12f4c7 (0xa4c8 bytes) 000008[1] 1779698207.650475 write 0x929000 thru 0x933e6f (0xae70 bytes) truncating to largest ever: 0x33f175 000009[1] 1779698207.714302 trunc from 0x933e70 to 0x33f174 (0x5f4cfb bytes) truncating to largest ever: 0x6a45fe 000010[0] 1779698207.772043 trunc from 0x33f175 to 0x6a45fd (0x365489 bytes) 000014[0] 1779698207.989125 mapread 0x4ef000 thru 0x4ef969 (0x96a bytes) 000017[0] 1779698208.060463 mapwrite 0x6b2000 thru 0x6b5dc5 (0x3dc6 bytes) truncating to largest ever: 0x7ee47e 000022[0] 1779698208.334283 trunc from 0x6b5dc6 to 0x7ee47d (0x1386b8 bytes) 000026[0] 1779698208.434399 mapread 0x2a4000 thru 0x2af9b7 (0xb9b8 bytes) 000027[0] 1779698208.511322 mapwrite 0x97f000 thru 0x9853cb (0x63cc bytes) 000029[1] 1779698208.722111 read 0x564000 thru 0x573d20 (0xfd21 bytes) 000030[0] 1779698208.798934 read 0x084000 thru 0x08514f (0x1150 bytes) 000031[0] 1779698208.807035 write 0x2d7000 thru 0x2da5a2 (0x35a3 bytes) 000032[0] 1779698208.845307 mapwrite 0x35c000 thru 0x35f71d (0x371e bytes) 000037[0] 1779698208.989194 mapread 0x0db000 thru 0x0e7315 (0xc316 bytes) 000040[0] 1779698209.057020 trunc from 0x9853cc to 0x512ea8 (0x472523 bytes) 000041[1] 1779698209.106811 read 0x4c6000 thru 0x4cdf3e (0x7f3f bytes) 000043[0] 1779698209.144713 read 0x41d000 thru 0x42c464 (0xf465 bytes) 000045[1] 1779698209.156274 mapread 0x4ff000 thru 0x500de4 (0x1de5 bytes) 000046[1] 1779698209.168239 write 0x536000 thru 0x54086a (0xa86b bytes) 000048[0] 1779698209.209049 mapwrite 0x21f000 thru 0x22e5ab (0xf5ac bytes) truncating to largest ever: 0x931dee 000049[1] 1779698209.389302 trunc from 0x54086b to 0x931ded (0x3f1583 bytes) 000050[1] 1779698209.451550 mapread 0x7ef000 thru 0x7f169a (0x269b bytes) 000052[0] 1779698209.594061 read 0x807000 thru 0x8097be (0x27bf bytes) 000053[0] 1779698209.625290 trunc from 0x931dee to 0x358609 (0x5d97e4 bytes) 000054[1] 1779698209.672871 trunc from 0x35860a to 0x551840 (0x1f9237 bytes) 000057[0] 1779698209.748471 trunc from 0x551841 to 0x67f56d (0x12dd2d bytes) 000058[0] 1779698209.799704 write 0x029000 thru 0x029f53 (0xf54 bytes) 000061[0] 1779698209.866368 read 0x4cb000 thru 0x4d201f (0x7020 bytes) 000063[1] 1779698209.915306 write 0x801000 thru 0x80f22f (0xe230 bytes) 000064[1] 1779698209.973443 mapwrite 0x672000 thru 0x672264 (0x265 bytes) 000065[0] 1779698210.054136 mapwrite 0x299000 thru 0x2a8c55 (0xfc56 bytes) 000071[0] 1779698210.312192 trunc from 0x80f230 to 0x4b07cb (0x35ea64 bytes) 000075[1] 1779698210.377343 mapwrite 0x84a000 thru 0x854f35 (0xaf36 bytes) 000077[0] 1779698210.513380 trunc from 0x854f36 to 0x2df84a (0x5756eb bytes) 000083[1] 1779698210.644540 read 0x273000 thru 0x27cef5 (0x9ef6 bytes) 000085[0] 1779698210.754784 trunc from 0x2df84b to 0x0fdf81 (0x1e18c9 bytes) 000090[1] 1779698210.884248 mapread 0x072000 thru 0x07646f (0x4470 bytes) 000093[0] 1779698210.967690 write 0x483000 thru 0x48c38b (0x938c bytes) 000096[1] 1779698211.082469 mapwrite 0x45d000 thru 0x46cb7f (0xfb80 bytes) 000098[1] 1779698211.274362 mapwrite 0x18e000 thru 0x19b818 (0xd819 bytes) 000099[0] 1779698211.445857 read 0x1e8000 thru 0x1edea3 (0x5ea4 bytes) 000100[0] 1779698211.479803 trunc from 0x48c38c to 0x00950b (0x482e80 bytes) 000101[0] 1779698211.522369 mapread 0x004000 thru 0x00861c (0x461d bytes) 000103[1] 1779698211.559874 read 0x005000 thru 0x008cdd (0x3cde bytes) 000104[0] 1779698211.596755 write 0x0df000 thru 0x0e51c2 (0x61c3 bytes) 000106[0] 1779698211.640319 read 0x081000 thru 0x0888cd (0x78ce bytes) 000109[0] 1779698211.737766 trunc from 0x0e51c3 to 0x5a9e12 (0x4c4c50 bytes) 000110[1] 1779698211.790091 write 0x113000 thru 0x11f28b (0xc28c bytes) 000113[0] 1779698211.912676 trunc from 0x5a9e13 to 0x6819b8 (0xd7ba6 bytes) 000114[0] 1779698211.993860 read 0x5b9000 thru 0x5be685 (0x5686 bytes) 000121[0] 1779698212.123897 mapread 0x34b000 thru 0x35462a (0x962b bytes) 000123[0] 1779698212.168253 trunc from 0x6819b9 to 0x582e87 (0xfeb31 bytes) 000124[0] 1779698212.222885 mapwrite 0x658000 thru 0x661d33 (0x9d34 bytes) 000126[1] 1779698212.359942 write 0x86f000 thru 0x873443 (0x4444 bytes) 000127[0] 1779698212.396740 trunc from 0x873444 to 0x323837 (0x54fc0c bytes) 000128[1] 1779698212.473755 mapwrite 0x5fd000 thru 0x5ff10f (0x2110 bytes) 000129[1] 1779698212.702227 read 0x4b2000 thru 0x4be277 (0xc278 bytes) 000130[0] 1779698212.744014 mapread 0x4ee000 thru 0x4ef50a (0x150b bytes) 000132[1] 1779698212.798358 read 0x57c000 thru 0x57c248 (0x249 bytes) 000134[0] 1779698212.805755 mapwrite 0x7dd000 thru 0x7eaa41 (0xda42 bytes) 000138[1] 1779698213.067874 read 0x4ae000 thru 0x4b986b (0xb86c bytes) 000142[1] 1779698213.111800 trunc from 0x7eaa42 to 0x0d670f (0x714332 bytes) 000143[0] 1779698213.167544 mapwrite 0x98e000 thru 0x98ef6f (0xf70 bytes) 000146[0] 1779698213.318969 read 0x76d000 thru 0x7750bd (0x80be bytes) 000148[0] 1779698213.379076 read 0x07d000 thru 0x07f5e9 (0x25ea bytes) 000150[0] 1779698213.411934 read 0x135000 thru 0x13846e (0x346f bytes) 000153[0] 1779698213.494283 read 0x65d000 thru 0x65d9ba (0x9bb bytes) 000156[1] 1779698213.523495 trunc from 0x98ef70 to 0x038e97 (0x9560d8 bytes) 000159[1] 1779698213.629854 trunc from 0x038e98 to 0x2f4b85 (0x2bbcee bytes) 000161[0] 1779698213.658517 write 0x595000 thru 0x5a1144 (0xc145 bytes) 000162[1] 1779698213.684865 mapwrite 0x21d000 thru 0x229ac2 (0xcac3 bytes) 000164[0] 1779698213.885406 read 0x057000 thru 0x05cbe4 (0x5be5 bytes) 000166[1] 1779698213.993494 mapwrite 0x376000 thru 0x376257 (0x258 bytes) 000168[1] 1779698214.258312 mapwrite 0x1ac000 thru 0x1b0bfb (0x4bfc bytes) 000169[1] 1779698214.426042 trunc from 0x5a1145 to 0x010ff6 (0x59014e bytes) 000170[1] 1779698214.464271 read 0x008000 thru 0x010d6d (0x8d6e bytes) 000172[0] 1779698214.536370 mapread 0x00e000 thru 0x010820 (0x2821 bytes) 000173[0] 1779698214.595105 mapwrite 0x6ce000 thru 0x6d5eeb (0x7eec bytes) 000174[1] 1779698214.820293 trunc from 0x6d5eec to 0x0e59ba (0x5f0531 bytes) 000175[0] 1779698214.901039 read 0x012000 thru 0x016750 (0x4751 bytes) 000178[1] 1779698214.987417 mapwrite 0x793000 thru 0x7985cd (0x55ce bytes) 000181[0] 1779698215.470795 mapread 0x415000 thru 0x4186d6 (0x36d7 bytes) 000182[0] 1779698215.557451 read 0x326000 thru 0x3344a9 (0xe4aa bytes) 000186[1] 1779698215.580326 read 0x4e4000 thru 0x4f193c (0xd93d bytes) 000188[0] 1779698215.594008 mapwrite 0x65c000 thru 0x665606 (0x9607 bytes) 000189[0] 1779698215.715637 mapread 0x37e000 thru 0x384363 (0x6364 bytes) 000190[1] 1779698215.738058 read 0x1d7000 thru 0x1e1f5d (0xaf5e bytes) 000191[1] 1779698215.770440 read 0x292000 thru 0x2a0eef (0xeef0 bytes) 000194[1] 1779698215.783914 mapread 0x31f000 thru 0x320095 (0x1096 bytes) 000197[0] 1779698215.800787 mapread 0x20f000 thru 0x21bb28 (0xcb29 bytes) 000200[0] 1779698215.843776 mapread 0x068000 thru 0x074042 (0xc043 bytes) 000202[0] 1779698215.895773 read 0x480000 thru 0x4813b3 (0x13b4 bytes) 000203[0] 1779698215.903917 read 0x015000 thru 0x022aaf (0xdab0 bytes) 000204[1] 1779698215.926447 write 0x299000 thru 0x29ee70 (0x5e71 bytes) 000205[0] 1779698215.985221 read 0x57a000 thru 0x57fc59 (0x5c5a bytes) 000208[0] 1779698216.033156 mapwrite 0x8a9000 thru 0x8ad079 (0x407a bytes) 000210[0] 1779698216.195280 trunc from 0x8ad07a to 0x0376f9 (0x875980 bytes) 000211[0] 1779698216.245221 write 0x181000 thru 0x18bd6f (0xad70 bytes) 000212[1] 1779698216.256188 trunc from 0x18bd70 to 0x0d29dc (0xb9393 bytes) 000213[1] 1779698216.319899 write 0x95b000 thru 0x962cc9 (0x7cca bytes) 000214[1] 1779698216.333916 mapwrite 0x8a6000 thru 0x8ade63 (0x7e64 bytes) 000215[0] 1779698216.483333 mapwrite 0x0f6000 thru 0x10346a (0xd46b bytes) 000220[0] 1779698216.695322 mapwrite 0x3d6000 thru 0x3e4a59 (0xea5a bytes) 000222[0] 1779698216.856028 mapwrite 0x0e2000 thru 0x0e4c0a (0x2c0b bytes) 000226[0] 1779698217.067173 mapwrite 0x8c5000 thru 0x8caa8c (0x5a8d bytes) 000228[1] 1779698217.202985 mapwrite 0x2bf000 thru 0x2c1f66 (0x2f67 bytes) 000229[1] 1779698217.416529 read_OD 0x7d7000 thru 0x7dffff (0x9000 bytes) 000233[0] 1779698217.468782 mapwrite 0x2e6000 thru 0x2eebe0 (0x8be1 bytes) 000236[1] 1779698217.769866 trunc from 0x962cca to 0x2f94f9 (0x6697d0 bytes) 000238[0] 1779698217.831252 trunc from 0x2f94fa to 0x6d43e5 (0x3daeec bytes) 000243[1] 1779698217.961842 mapwrite 0x191000 thru 0x196e76 (0x5e77 bytes) 000244[0] 1779698218.119122 mapread 0x319000 thru 0x327f66 (0xef67 bytes) 000245[1] 1779698218.181450 trunc from 0x6d43e6 to 0x6d23fb (0x1fea bytes) 000247[0] 1779698218.240109 write 0x18d000 thru 0x198d1c (0xbd1d bytes) 000248[0] 1779698218.287622 mapwrite 0x4ce000 thru 0x4dcfae (0xefaf bytes) 000249[1] 1779698218.405845 write_OD 0x2c8000 thru 0x2cefff (0x7000 bytes) 000250[0] 1779698218.484402 read 0x598000 thru 0x59d101 (0x5102 bytes) 000251[1] 1779698218.506447 read_OD 0x36d000 thru 0x371fff (0x5000 bytes) 000256[0] 1779698218.528445 trunc from 0x6d23fc to 0x195aa9 (0x53c952 bytes) 000257[0] 1779698218.595746 write 0x271000 thru 0x27fdbe (0xedbf bytes) 000258[1] 1779698218.603563 write_OD 0x2a5000 thru 0x2b0fff (0xc000 bytes) 000259[0] 1779698218.688210 write 0x664000 thru 0x6711f0 (0xd1f1 bytes) 000262[0] 1779698218.721711 read 0x3a2000 thru 0x3a8ce7 (0x6ce8 bytes) 000264[0] 1779698218.774090 mapread 0x2b0000 thru 0x2bb98e (0xb98f bytes) 000266[1] 1779698218.841481 write_OD 0x088000 thru 0x089fff (0x2000 bytes) 000269[0] 1779698218.923697 mapwrite 0x3b2000 thru 0x3bd259 (0xb25a bytes) 000270[0] 1779698219.116830 write 0x5c7000 thru 0x5c9417 (0x2418 bytes) 000273[1] 1779698219.143313 mapread 0x547000 thru 0x54dcea (0x6ceb bytes) 000280[1] 1779698219.206467 trunc from 0x6711f1 to 0x7d6eec (0x165cfc bytes) 000281[1] 1779698219.247537 mapwrite 0x9a5000 thru 0x9ade96 (0x8e97 bytes) 000282[0] 1779698219.373983 mapwrite 0x6fa000 thru 0x700505 (0x6506 bytes) 000283[0] 1779698219.513571 read 0x851000 thru 0x858d63 (0x7d64 bytes) 000284[1] 1779698219.545738 write_OD 0x8ae000 thru 0x8bbfff (0xe000 bytes) 000285[1] 1779698219.618569 mapwrite 0x09d000 thru 0x0a9ff8 (0xcff9 bytes) 000292[0] 1779698219.911143 write 0x053000 thru 0x05e2c8 (0xb2c9 bytes) 000294[1] 1779698219.975000 mapread 0x8df000 thru 0x8eed8b (0xfd8c bytes) 000296[1] 1779698220.053909 trunc from 0x9ade97 to 0x205dd6 (0x7a80c0 bytes) 000297[1] 1779698220.096449 write_OD 0x08d000 thru 0x08efff (0x2000 bytes) 000298[0] 1779698220.196431 mapwrite 0x301000 thru 0x30de68 (0xce69 bytes) 000299[1] 1779698220.415632 read_OD 0x08f000 thru 0x09dfff (0xf000 bytes) 000301[1] 1779698220.497875 read_OD 0x146000 thru 0x152fff (0xd000 bytes) 000302[0] 1779698220.517829 read 0x216000 thru 0x21ea22 (0x8a23 bytes) 000303[1] 1779698220.596749 write_OD 0x705000 thru 0x70ffff (0xb000 bytes) 000306[1] 1779698220.695563 mapwrite 0x3e0000 thru 0x3e55b3 (0x55b4 bytes) 000308[0] 1779698220.918928 mapwrite 0x902000 thru 0x90b92f (0x9930 bytes) 000309[1] 1779698221.206231 trunc from 0x90b930 to 0x043a4d (0x8c7ee2 bytes) 000310[0] 1779698221.266554 mapread 0x028000 thru 0x0322b6 (0xa2b7 bytes) 000311[0] 1779698221.415887 write 0x7ee000 thru 0x7f95d0 (0xb5d1 bytes) 000313[1] 1779698221.464914 write_OD 0x6c7000 thru 0x6ccfff (0x6000 bytes) 000315[1] 1779698221.611690 read_OD 0x0da000 thru 0x0ddfff (0x4000 bytes) 000317[0] 1779698221.630227 trunc from 0x7f95d1 to 0x25a541 (0x59f08f bytes) 000319[0] 1779698221.737924 trunc from 0x25a542 to 0x68da58 (0x433517 bytes) 000321[1] 1779698221.792766 mapread 0x04d000 thru 0x05acf0 (0xdcf1 bytes) 000322[0] 1779698221.868135 write 0x239000 thru 0x241fc3 (0x8fc4 bytes) 000323[1] 1779698221.907014 read_OD 0x452000 thru 0x452fff (0x1000 bytes) 000324[0] 1779698221.948633 read 0x56b000 thru 0x575d00 (0xad01 bytes) 000325[1] 1779698221.963691 trunc from 0x68da59 to 0x131897 (0x55c1c1 bytes) 000326[1] 1779698222.025870 trunc from 0x131898 to 0x88901b (0x757784 bytes) 000329[0] 1779698222.093461 trunc from 0x88901c to 0x13bbd8 (0x74d443 bytes) 000330[1] 1779698222.130196 mapread 0x0a7000 thru 0x0b0625 (0x9626 bytes) 000331[1] 1779698222.189538 write_OD 0x452000 thru 0x45cfff (0xb000 bytes) 000333[0] 1779698222.270399 mapread 0x10f000 thru 0x10ffb2 (0xfb3 bytes) 000334[0] 1779698222.281118 mapwrite 0x343000 thru 0x344729 (0x172a bytes) 000335[0] 1779698222.383160 read 0x252000 thru 0x25b696 (0x9697 bytes) 000339[0] 1779698222.459578 trunc from 0x45d000 to 0x599548 (0x13c549 bytes) 000340[1] 1779698222.485915 mapread 0x4c2000 thru 0x4cb6dc (0x96dd bytes) 000342[1] 1779698222.531460 read_OD 0x149000 thru 0x149fff (0x1000 bytes) 000343[1] 1779698222.540422 trunc from 0x599549 to 0x27c6ed (0x31ce5b bytes) 000344[0] 1779698222.618784 trunc from 0x27c6ee to 0x748678 (0x4cbf8b bytes) 000346[0] 1779698222.669251 mapread 0x2a5000 thru 0x2aec5b (0x9c5c bytes) 000347[1] 1779698222.730845 mapwrite 0x13d000 thru 0x146301 (0x9302 bytes) 000348[0] 1779698222.992665 trunc from 0x748679 to 0x76051e (0x17ea6 bytes) 000349[0] 1779698223.094924 write 0x8cf000 thru 0x8d0e12 (0x1e13 bytes) 000351[1] 1779698223.148872 mapwrite 0x309000 thru 0x30ce16 (0x3e17 bytes) 000353[1] 1779698223.446826 read_OD 0x506000 thru 0x509fff (0x4000 bytes) 000355[1] 1779698223.491315 trunc from 0x8d0e13 to 0x8a7146 (0x29ccc bytes) 000358[0] 1779698223.630281 mapwrite 0x793000 thru 0x79d9ed (0xa9ee bytes) 000360[0] 1779698223.758793 trunc from 0x8a7147 to 0x5c06e5 (0x2e6a61 bytes) 000362[1] 1779698223.810240 mapwrite 0x208000 thru 0x212c0d (0xac0e bytes) 000363[1] 1779698223.908412 write_OD 0x147000 thru 0x149fff (0x3000 bytes) 000364[1] 1779698223.974407 read_OD 0x53b000 thru 0x540fff (0x6000 bytes) 000367[1] 1779698223.997846 read_OD 0x58d000 thru 0x594fff (0x8000 bytes) 000368[0] 1779698224.009761 read 0x1e8000 thru 0x1ed056 (0x5057 bytes) 000370[0] 1779698224.026932 read 0x20d000 thru 0x21aa61 (0xda62 bytes) 000371[1] 1779698224.041486 read_OD 0x289000 thru 0x28efff (0x6000 bytes) 000372[0] 1779698224.054806 read 0x3dd000 thru 0x3df87b (0x287c bytes) 000375[0] 1779698224.064204 write 0x32c000 thru 0x32c8b8 (0x8b9 bytes) 000377[0] 1779698224.113232 read 0x509000 thru 0x510dd9 (0x7dda bytes) 000380[0] 1779698224.170104 read 0x5a8000 thru 0x5b12f8 (0x92f9 bytes) 000381[0] 1779698224.183890 mapread 0x04e000 thru 0x05c5df (0xe5e0 bytes) 000383[1] 1779698224.244894 mapwrite 0x3e8000 thru 0x3f1bd7 (0x9bd8 bytes) 000384[1] 1779698224.442798 mapwrite 0x56d000 thru 0x57c71e (0xf71f bytes) 000389[0] 1779698224.640654 mapread 0x4f4000 thru 0x4fa27f (0x6280 bytes) 000397[0] 1779698224.780738 mapread 0x44f000 thru 0x44f979 (0x97a bytes) 000399[1] 1779698224.808855 trunc from 0x5c06e6 to 0x79f3a3 (0x1decbe bytes) 000400[0] 1779698224.914539 read 0x29d000 thru 0x2acd46 (0xfd47 bytes) 000404[1] 1779698225.016498 mapwrite 0x3b9000 thru 0x3bdd3c (0x4d3d bytes) 000405[0] 1779698225.210902 mapwrite 0x727000 thru 0x72784e (0x84f bytes) 000408[0] 1779698225.373740 trunc from 0x79f3a4 to 0x64450e (0x15ae95 bytes) 000409[0] 1779698225.417196 trunc from 0x64450f to 0x26d109 (0x3d7405 bytes) 000410[1] 1779698225.446348 trunc from 0x26d10a to 0x2e7d2f (0x7ac26 bytes) 000414[0] 1779698225.555698 write 0x315000 thru 0x31e48f (0x9490 bytes) 000415[1] 1779698225.577614 read 0x02e000 thru 0x031299 (0x329a bytes) 000416[1] 1779698225.615384 mapwrite 0x852000 thru 0x85e3f7 (0xc3f8 bytes) 000417[1] 1779698225.906053 trunc from 0x85e3f8 to 0x344a79 (0x51997e bytes) 000418[1] 1779698225.929545 write 0x682000 thru 0x6882be (0x62bf bytes) 000419[0] 1779698225.965161 read 0x4b5000 thru 0x4b72e7 (0x22e8 bytes) 000421[1] 1779698226.012465 write 0x54f000 thru 0x557f62 (0x8f63 bytes) 000422[0] 1779698226.071370 trunc from 0x6882bf to 0x81d685 (0x1953c7 bytes) 000425[0] 1779698226.317563 mapwrite 0x963000 thru 0x967e92 (0x4e93 bytes) 000430[1] 1779698226.692169 read 0x0e6000 thru 0x0e7232 (0x1233 bytes) 000432[1] 1779698226.740209 write 0x894000 thru 0x8a25d8 (0xe5d9 bytes) 000433[0] 1779698226.775203 read 0x54f000 thru 0x54f8fa (0x8fb bytes) 000434[0] 1779698226.837820 mapwrite 0x563000 thru 0x56c53f (0x9540 bytes) 000438[0] 1779698227.001458 write 0x75f000 thru 0x766435 (0x7436 bytes) 000439[0] 1779698227.050036 read 0x5b1000 thru 0x5b6cd6 (0x5cd7 bytes) 000444[0] 1779698227.117691 mapread 0x42b000 thru 0x42c305 (0x1306 bytes) 000445[0] 1779698227.169211 write 0x482000 thru 0x48aa7c (0x8a7d bytes) 000446[1] 1779698227.182830 trunc from 0x967e93 to 0x473742 (0x4f4750 bytes) 000448[1] 1779698227.276934 write 0x8da000 thru 0x8dde1e (0x3e1f bytes) 000450[1] 1779698227.290882 mapwrite 0x9e6000 thru 0x9ec294 (0x6295 bytes) 000451[1] 1779698227.541748 read 0x08b000 thru 0x097aef (0xcaf0 bytes) 000454[1] 1779698227.625168 mapread 0x856000 thru 0x85e821 (0x8822 bytes) 000455[0] 1779698227.667213 mapwrite 0x38a000 thru 0x38cd73 (0x2d74 bytes) 000459[1] 1779698227.831510 mapread 0x2cf000 thru 0x2dce04 (0xde05 bytes) 000461[0] 1779698228.019473 mapwrite 0x3d7000 thru 0x3e4276 (0xd277 bytes) 000462[1] 1779698228.277907 mapread 0x25b000 thru 0x263c6e (0x8c6f bytes) 000463[1] 1779698228.345311 trunc from 0x9ec295 to 0x50cd13 (0x4df581 bytes) 000464[1] 1779698228.448111 read 0x4a0000 thru 0x4a2034 (0x2035 bytes) 000466[1] 1779698228.467928 write 0x2d6000 thru 0x2e25d0 (0xc5d1 bytes) 000467[1] 1779698228.491177 mapwrite 0x824000 thru 0x82c9f8 (0x89f9 bytes) 000468[0] 1779698228.677890 read 0x4b0000 thru 0x4bfe7d (0xfe7e bytes) 000469[0] 1779698228.762120 write 0x615000 thru 0x621fa9 (0xcfaa bytes) 000472[0] 1779698228.887577 trunc from 0x82c9f9 to 0x4c37a0 (0x369258 bytes) 000474[0] 1779698229.035364 mapread 0x370000 thru 0x37c696 (0xc697 bytes) 000475[0] 1779698229.167441 write 0x389000 thru 0x38f52f (0x6530 bytes) 000476[1] 1779698229.218264 mapwrite 0x04e000 thru 0x050ca5 (0x2ca6 bytes) 000477[0] 1779698229.489479 read 0x29b000 thru 0x2a6e1a (0xbe1b bytes) 000478[1] 1779698229.553099 trunc from 0x4c37a1 to 0x11506c (0x3ae734 bytes) 000479[1] 1779698229.654428 write 0x398000 thru 0x3a5130 (0xd131 bytes) 000480[0] 1779698229.677170 mapread 0x2ac000 thru 0x2b928f (0xd290 bytes) 000482[0] 1779698229.779786 mapwrite 0x711000 thru 0x720048 (0xf049 bytes) 000483[1] 1779698230.099727 read 0x0ca000 thru 0x0cc372 (0x2373 bytes) 000485[0] 1779698230.168920 trunc from 0x720049 to 0x20f8fe (0x51074a bytes) 000488[1] 1779698230.270319 read 0x1bb000 thru 0x1c4f76 (0x9f77 bytes) truncating to largest ever: 0x9a9422 000495[1] 1779698230.354327 trunc from 0x20f8ff to 0x9a9421 (0x799b23 bytes) 000496[0] 1779698230.458459 read 0x1d8000 thru 0x1e4ef8 (0xcef9 bytes) 000499[1] 1779698230.615661 trunc from 0x9a9422 to 0x874515 (0x134f0c bytes) All operations completed A-OK! PASS 16a (67s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 04:37:18 (1779698238) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.531514 s, 15.8 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.455496 s, 18.4 MB/s Chance of close/open is 1 in 50 Seed set to 8241 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read skipping zero size read 000008[1] 1779698242.834715 write 0x9888c3 thru 0x98a2d9 (0x1a17 bytes) 000009[0] 1779698242.900198 write 0x1e4012 thru 0x1e4336 (0x325 bytes) 000010[1] 1779698242.986206 mapwrite 0x99e0de thru 0x99f9b5 (0x18d8 bytes) truncating to largest ever: 0x534f0 000011[0] 1779698243.203493 trunc from 0x99f9b6 to 0x0534ef (0x94c4c6 bytes) 000013[0] 1779698243.282672 mapread 0x00de3d thru 0x00ef2a (0x10ee bytes) 000018[0] 1779698243.363362 mapread 0x034aff thru 0x03579e (0xca0 bytes) truncating to largest ever: 0x7d8c62 000019[0] 1779698243.374951 trunc from 0x0534f0 to 0x7d8c61 (0x785772 bytes) 000020[0] 1779698243.456180 mapwrite 0x895a0f thru 0x896297 (0x889 bytes) 000021[1] 1779698243.716472 read 0x088c1f thru 0x088f5b (0x33d bytes) 000022[0] 1779698243.764687 mapread 0x761820 thru 0x762546 (0xd27 bytes) 000025[0] 1779698243.835575 write 0x8bc77d thru 0x8bdb79 (0x13fd bytes) 000026[1] 1779698243.895531 write 0x34d81d thru 0x34e671 (0xe55 bytes) 000028[0] 1779698243.967217 read 0x7959d7 thru 0x79607c (0x6a6 bytes) 000030[1] 1779698244.023392 mapwrite 0x52c81f thru 0x52c9df (0x1c1 bytes) 000032[0] 1779698244.172722 write 0x726145 thru 0x72654f (0x40b bytes) 000035[1] 1779698244.254370 mapwrite 0x8a8058 thru 0x8a9994 (0x193d bytes) 000039[0] 1779698244.429998 trunc from 0x8bdb7a to 0x2fcedf (0x5c0c9a bytes) 000040[1] 1779698244.535356 mapwrite 0x7a6386 thru 0x7a688a (0x505 bytes) 000041[0] 1779698244.677723 write 0x19df1a thru 0x19e184 (0x26b bytes) 000042[0] 1779698244.771650 trunc from 0x7a688b to 0x327186 (0x47f704 bytes) 000045[0] 1779698244.831432 write 0x0c4374 thru 0x0c5d39 (0x19c6 bytes) 000046[0] 1779698244.859877 trunc from 0x327187 to 0x23e885 (0xe8901 bytes) 000050[0] 1779698244.990147 trunc from 0x23e886 to 0x06a8b5 (0x1d3fd0 bytes) 000051[1] 1779698245.037543 read 0x020b0c thru 0x021cf8 (0x11ed bytes) 000052[1] 1779698245.108283 mapwrite 0x3f17a0 thru 0x3f28b9 (0x111a bytes) 000053[1] 1779698245.316167 write 0x1bc1f4 thru 0x1bc508 (0x315 bytes) 000054[1] 1779698245.342563 mapwrite 0x8dbc0e thru 0x8ddb17 (0x1f0a bytes) 000056[1] 1779698245.632619 trunc from 0x8ddb18 to 0x31ed05 (0x5bee12 bytes) 000058[1] 1779698245.705971 mapread 0x1fdd33 thru 0x1fec1a (0xee8 bytes) 000059[0] 1779698245.707908 mapwrite 0x630de1 thru 0x631c2d (0xe4d bytes) 000061[0] 1779698245.913612 mapwrite 0x25bece thru 0x25dd04 (0x1e37 bytes) 000063[0] 1779698246.039521 read 0x55dfde thru 0x55e775 (0x798 bytes) 000067[0] 1779698246.117222 write 0x4d4209 thru 0x4d52bf (0x10b7 bytes) 000068[0] 1779698246.131193 trunc from 0x631c2e to 0x51861b (0x119612 bytes) 000069[0] 1779698246.162506 mapread 0x205dd6 thru 0x207afa (0x1d25 bytes) truncating to largest ever: 0x7ee7d3 000071[0] 1779698246.219650 trunc from 0x51861c to 0x7ee7d2 (0x2d61b7 bytes) 000074[1] 1779698246.305230 write 0x2af59e thru 0x2b0461 (0xec4 bytes) 000078[0] 1779698246.411483 read 0x7bfce5 thru 0x7c1aac (0x1dc8 bytes) 000079[0] 1779698246.438341 write 0x643b63 thru 0x644be4 (0x1082 bytes) 000081[0] 1779698246.495893 trunc from 0x7ee7d3 to 0x0a51f6 (0x7495dc bytes) 000086[0] 1779698246.586679 trunc from 0x0a51f7 to 0x2f9f92 (0x254d9c bytes) 000087[1] 1779698246.645191 read 0x22c1df thru 0x22defa (0x1d1c bytes) 000088[0] 1779698246.746992 trunc from 0x2f9f93 to 0x31cdaf (0x22e1d bytes) 000089[1] 1779698246.891802 mapread 0x2f0ac3 thru 0x2f2a01 (0x1f3f bytes) 000090[1] 1779698247.010432 write 0x6b47e5 thru 0x6b4c95 (0x4b1 bytes) truncating to largest ever: 0x9159bf 000091[0] 1779698247.062810 trunc from 0x6b4c96 to 0x9159be (0x260d29 bytes) 000097[0] 1779698247.306680 trunc from 0x9159bf to 0x6a107a (0x274944 bytes) 000099[0] 1779698247.398453 mapread 0x363c4d thru 0x363f85 (0x339 bytes) 000100[1] 1779698247.437410 trunc from 0x6a107b to 0x1f88e4 (0x4a8796 bytes) 000101[0] 1779698247.490512 trunc from 0x1f88e5 to 0x09ee88 (0x159a5c bytes) 000103[0] 1779698247.610383 write 0x1affe2 thru 0x1b027b (0x29a bytes) 000104[0] 1779698247.614499 read 0x0e5c5d thru 0x0e7169 (0x150d bytes) 000105[0] 1779698247.673214 read 0x03dea6 thru 0x03ef9b (0x10f6 bytes) 000106[0] 1779698247.674418 read 0x12904a thru 0x129f51 (0xf08 bytes) 000109[0] 1779698247.706721 write 0x964091 thru 0x965e87 (0x1df7 bytes) 000110[0] 1779698247.716419 trunc from 0x965e88 to 0x1e8a34 (0x77d453 bytes) 000112[0] 1779698247.743343 write 0x4365b7 thru 0x436c9e (0x6e8 bytes) 000113[1] 1779698247.780347 read 0x016659 thru 0x0170f2 (0xa9a bytes) 000114[1] 1779698247.835581 mapwrite 0x66d953 thru 0x66f2a0 (0x194e bytes) 000116[1] 1779698248.030572 mapread 0x3ba9cd thru 0x3bac84 (0x2b8 bytes) 000118[0] 1779698248.114328 trunc from 0x66f2a1 to 0x4cc216 (0x1a308a bytes) 000122[0] 1779698248.294525 trunc from 0x4cc217 to 0x053ef8 (0x47831e bytes) 000124[0] 1779698248.375436 read 0x02c8bb thru 0x02e370 (0x1ab6 bytes) 000126[0] 1779698248.466574 read 0x03226c thru 0x032f8f (0xd24 bytes) 000127[1] 1779698248.501221 write 0x5d6b7b thru 0x5d7b08 (0xf8e bytes) 000130[0] 1779698248.588424 write 0x2dd80d thru 0x2ddc0f (0x403 bytes) 000131[0] 1779698248.618457 mapwrite 0x45ce58 thru 0x45cf45 (0xee bytes) 000133[0] 1779698248.755707 trunc from 0x5d7b09 to 0x4ca933 (0x10d1d5 bytes) 000135[1] 1779698248.826113 write 0x8a41b0 thru 0x8a4318 (0x169 bytes) 000137[1] 1779698248.926526 trunc from 0x8a4319 to 0x4dad8e (0x3c958a bytes) 000138[0] 1779698249.005721 mapread 0x23a4cd thru 0x23a621 (0x155 bytes) 000141[1] 1779698249.160432 mapread 0x37ce90 thru 0x37d7f4 (0x965 bytes) 000143[0] 1779698249.191371 read 0x3faae9 thru 0x3fbf4c (0x1464 bytes) 000146[0] 1779698249.214911 trunc from 0x4dad8f to 0x2c0d0b (0x21a083 bytes) 000148[0] 1779698249.342931 trunc from 0x2c0d0c to 0x826a45 (0x565d3a bytes) 000149[1] 1779698249.378028 write 0x25c19a thru 0x25c58b (0x3f2 bytes) 000150[1] 1779698249.417475 mapread 0x7d79cb thru 0x7d9276 (0x18ac bytes) 000153[0] 1779698249.538508 write 0x2d35f2 thru 0x2d52a0 (0x1caf bytes) 000154[0] 1779698249.571334 write 0x94a1be thru 0x94bf40 (0x1d83 bytes) 000157[0] 1779698249.656514 write 0x09a3be thru 0x09b088 (0xccb bytes) 000161[0] 1779698249.724132 mapwrite 0x3a9c6d thru 0x3ab93d (0x1cd1 bytes) 000165[1] 1779698249.820676 read 0x54150f thru 0x542d55 (0x1847 bytes) truncating to largest ever: 0x95aeb1 000166[0] 1779698249.846541 trunc from 0x94bf41 to 0x95aeb0 (0xef70 bytes) 000167[1] 1779698249.893877 write 0x3aee0f thru 0x3b004f (0x1241 bytes) 000168[0] 1779698249.951418 mapwrite 0x2e5dbb thru 0x2e67c9 (0xa0f bytes) 000171[0] 1779698250.230983 mapwrite 0x92d518 thru 0x92ddf3 (0x8dc bytes) 000172[1] 1779698250.492024 read 0x163857 thru 0x1649a4 (0x114e bytes) 000173[1] 1779698250.596205 trunc from 0x95aeb1 to 0x8194ee (0x1419c2 bytes) 000174[1] 1779698250.757490 mapwrite 0x1d250f thru 0x1d44b4 (0x1fa6 bytes) 000175[1] 1779698250.885538 mapwrite 0x278dd6 thru 0x279335 (0x560 bytes) 000176[1] 1779698250.962662 mapwrite 0x82452b thru 0x8257bb (0x1291 bytes) 000177[0] 1779698251.093563 read 0x339fa1 thru 0x33b89a (0x18fa bytes) 000178[0] 1779698251.128648 read 0x1892d6 thru 0x18a135 (0xe60 bytes) 000179[1] 1779698251.135313 read 0x1c3766 thru 0x1c561d (0x1eb8 bytes) 000180[0] 1779698251.144519 mapread 0x1543ef thru 0x155fe6 (0x1bf8 bytes) 000181[0] 1779698251.167816 mapwrite 0x9ad789 thru 0x9ae2d4 (0xb4c bytes) 000182[1] 1779698251.307803 trunc from 0x9ae2d5 to 0x738896 (0x275a3e bytes) 000186[1] 1779698251.486647 mapread 0x2b3029 thru 0x2b3f0e (0xee6 bytes) 000187[0] 1779698251.503463 mapread 0x0789dd thru 0x07a1b7 (0x17db bytes) 000192[1] 1779698251.622814 mapread 0x6adc1d thru 0x6aedb2 (0x1196 bytes) 000195[1] 1779698251.657150 mapread 0x3b1282 thru 0x3b149d (0x21c bytes) 000197[1] 1779698251.685452 mapread 0x0237d3 thru 0x0240c9 (0x8f7 bytes) 000198[1] 1779698251.720373 mapwrite 0x3e7ff7 thru 0x3e8565 (0x56f bytes) 000199[0] 1779698251.891275 read 0x7171cb thru 0x717e3d (0xc73 bytes) 000201[0] 1779698251.999449 mapwrite 0x6c06e0 thru 0x6c1981 (0x12a2 bytes) 000202[0] 1779698252.129644 mapwrite 0x01ccf5 thru 0x01ea2d (0x1d39 bytes) 000203[0] 1779698252.263129 mapwrite 0x292846 thru 0x292c68 (0x423 bytes) 000207[1] 1779698252.393624 write 0x4f5290 thru 0x4f5295 (0x6 bytes) 000208[0] 1779698252.434618 mapread 0x5391bf thru 0x53ae8a (0x1ccc bytes) 000209[0] 1779698252.498090 mapread 0x694be6 thru 0x695617 (0xa32 bytes) 000210[1] 1779698252.513887 mapread 0x3b4e55 thru 0x3b5dcd (0xf79 bytes) 000212[0] 1779698252.562699 write 0x1edd36 thru 0x1efb0b (0x1dd6 bytes) 000213[1] 1779698252.593183 mapwrite 0x0ffd53 thru 0x100614 (0x8c2 bytes) 000215[0] 1779698252.728536 mapwrite 0x42e8f0 thru 0x42f3eb (0xafc bytes) 000216[1] 1779698252.849129 read 0x0faa39 thru 0x0fb173 (0x73b bytes) 000220[0] 1779698252.925287 mapwrite 0x3b87e6 thru 0x3b88e2 (0xfd bytes) 000221[0] 1779698253.063821 read 0x376097 thru 0x3768ac (0x816 bytes) 000223[0] 1779698253.111518 read 0x5eca9b thru 0x5ed9af (0xf15 bytes) 000224[1] 1779698253.148542 mapread 0x5b4e2c thru 0x5b5837 (0xa0c bytes) 000227[1] 1779698253.244151 mapread 0x33e754 thru 0x33eb2f (0x3dc bytes) 000228[0] 1779698253.292326 write 0x2fbfc6 thru 0x2fd47a (0x14b5 bytes) 000230[1] 1779698253.339222 mapwrite 0x00211a thru 0x002f7f (0xe66 bytes) 000232[0] 1779698253.469813 mapread 0x4ebd9f thru 0x4ecf3d (0x119f bytes) 000233[0] 1779698253.494489 trunc from 0x738897 to 0x642c9f (0xf5bf7 bytes) 000235[0] 1779698253.632509 write 0x8ad351 thru 0x8ae31a (0xfca bytes) 000237[0] 1779698253.645467 read 0x66d869 thru 0x66f6af (0x1e47 bytes) 000240[0] 1779698253.716094 mapwrite 0x0ce28a thru 0x0ced7e (0xaf5 bytes) 000241[0] 1779698253.779946 mapread 0x36c43b thru 0x36ded1 (0x1a97 bytes) 000242[1] 1779698253.793091 read 0x76c085 thru 0x76dde8 (0x1d64 bytes) 000244[0] 1779698253.838238 trunc from 0x8ae31b to 0x18a6fe (0x723c1c bytes) truncating to largest ever: 0x9c0d80 000245[0] 1779698253.888725 trunc from 0x18a6ff to 0x9c0d7f (0x836681 bytes) 000248[0] 1779698253.980843 read 0x2b9c0c thru 0x2bac25 (0x101a bytes) 000249[0] 1779698254.021185 write 0x3f8ff5 thru 0x3f9245 (0x251 bytes) 000250[1] 1779698254.064968 write 0x213cd6 thru 0x214313 (0x63e bytes) 000253[0] 1779698254.166275 trunc from 0x9c0d80 to 0x3cb13d (0x5f5c42 bytes) 000254[0] 1779698254.238475 mapwrite 0x39868b thru 0x399ff1 (0x1967 bytes) 000256[0] 1779698254.353336 mapwrite 0x01a6e6 thru 0x01c343 (0x1c5e bytes) 000257[0] 1779698254.481112 mapwrite 0x3c493f thru 0x3c68cd (0x1f8f bytes) 000261[0] 1779698254.687255 mapwrite 0x1ea803 thru 0x1ec0f9 (0x18f7 bytes) 000262[0] 1779698254.800562 write 0x8a4e4b thru 0x8a5014 (0x1ca bytes) 000263[1] 1779698254.857579 read 0x0c1ab6 thru 0x0c2ddb (0x1326 bytes) 000264[0] 1779698254.925520 trunc from 0x8a5015 to 0x0830fc (0x821f18 bytes) 000270[0] 1779698255.093691 write 0x86fab6 thru 0x87178a (0x1cd5 bytes) 000271[1] 1779698255.160571 mapwrite 0x0f3470 thru 0x0f3c0c (0x79d bytes) 000272[0] 1779698255.421446 mapwrite 0x813ca8 thru 0x8141f0 (0x549 bytes) 000275[0] 1779698255.735737 mapwrite 0x083ce7 thru 0x084417 (0x731 bytes) 000277[0] 1779698256.104463 write 0x726bcb thru 0x727e2c (0x1262 bytes) 000278[1] 1779698256.205070 write 0x60b24a thru 0x60b414 (0x1cb bytes) 000281[0] 1779698256.307910 read 0x43e101 thru 0x43fd42 (0x1c42 bytes) 000282[1] 1779698256.372786 read 0x813419 thru 0x81436a (0xf52 bytes) 000283[0] 1779698256.396390 mapread 0x2b6874 thru 0x2b753e (0xccb bytes) 000284[1] 1779698256.424827 trunc from 0x87178b to 0x1d912e (0x69865c bytes) 000285[0] 1779698256.580204 write 0x61d028 thru 0x61daac (0xa85 bytes) 000289[0] 1779698256.791104 read 0x374af1 thru 0x375a27 (0xf37 bytes) 000291[0] 1779698256.901963 trunc from 0x61daad to 0x002b3d (0x61af6f bytes) 000294[0] 1779698257.063522 trunc from 0x002b3e to 0x0a3073 (0xa0536 bytes) 000295[0] 1779698257.123698 write 0x1e749e thru 0x1e9228 (0x1d8b bytes) 000297[0] 1779698257.236497 write 0x89a9eb thru 0x89ab44 (0x15a bytes) 000298[1] 1779698257.246522 write 0x13170e thru 0x131d51 (0x644 bytes) 000300[1] 1779698257.366657 trunc from 0x89ab45 to 0x6c000a (0x1dab3a bytes) 000302[0] 1779698257.564893 write 0x62bb29 thru 0x62c277 (0x74f bytes) 000304[0] 1779698257.631305 write 0x3aca9d thru 0x3ad947 (0xeab bytes) 000305[1] 1779698257.668036 read 0x19f208 thru 0x19f43f (0x238 bytes) 000307[1] 1779698257.740632 mapread 0x199ada thru 0x19a5bd (0xae4 bytes) 000308[1] 1779698257.790565 write 0x09a67a thru 0x09a68c (0x13 bytes) 000315[0] 1779698257.987571 trunc from 0x6c000b to 0x8f15e4 (0x2315da bytes) 000316[0] 1779698258.067848 write 0x5186d6 thru 0x519479 (0xda4 bytes) 000317[0] 1779698258.142270 mapread 0x60a67f thru 0x60a9c2 (0x344 bytes) 000320[1] 1779698258.318732 mapread 0x4271f6 thru 0x427fd5 (0xde0 bytes) 000321[1] 1779698258.381522 write 0x74d290 thru 0x74ef50 (0x1cc1 bytes) 000322[1] 1779698258.438608 mapread 0x00d7ca thru 0x00f1cc (0x1a03 bytes) 000324[0] 1779698258.528181 write 0x7f1add thru 0x7f292b (0xe4f bytes) 000329[0] 1779698258.630813 mapread 0x4e7919 thru 0x4e825b (0x943 bytes) truncating to largest ever: 0x9cae7c 000330[1] 1779698258.686535 trunc from 0x8f15e5 to 0x9cae7b (0xd9897 bytes) 000331[0] 1779698258.817710 write 0x5132e7 thru 0x514776 (0x1490 bytes) 000332[1] 1779698258.868268 trunc from 0x9cae7c to 0x3f589f (0x5d55dc bytes) 000333[1] 1779698258.912552 read 0x0c2a2e thru 0x0c2b62 (0x135 bytes) 000337[1] 1779698259.199396 write 0x9d52d4 thru 0x9d72b7 (0x1fe4 bytes) 000338[1] 1779698259.239531 write 0x3fae77 thru 0x3faf80 (0x10a bytes) 000339[1] 1779698259.371832 write 0x2fbf96 thru 0x2fc4db (0x546 bytes) 000340[1] 1779698259.417299 mapwrite 0x384411 thru 0x38608d (0x1c7d bytes) 000341[1] 1779698259.544975 trunc from 0x9d72b8 to 0x75a8b4 (0x27ca03 bytes) 000342[1] 1779698259.666489 read 0x5cced6 thru 0x5ced4a (0x1e75 bytes) 000344[1] 1779698259.693561 write 0x63dc7c thru 0x63de7b (0x200 bytes) 000346[1] 1779698259.698361 read 0x54e855 thru 0x54efff (0x7ab bytes) 000347[1] 1779698259.699185 trunc from 0x75a8b5 to 0x3397bc (0x4210f8 bytes) 000349[1] 1779698259.751214 read 0x1ee48a thru 0x1ee81c (0x393 bytes) 000350[1] 1779698259.806843 mapread 0x269c29 thru 0x26a99d (0xd75 bytes) 000356[1] 1779698259.976554 write 0x9e7391 thru 0x9e7caa (0x91a bytes) 000357[0] 1779698259.988227 mapread 0x9d34e3 thru 0x9d4588 (0x10a6 bytes) 000359[1] 1779698260.170162 mapwrite 0x1e91f3 thru 0x1ea777 (0x1585 bytes) 000361[1] 1779698260.416848 mapread 0x348061 thru 0x34815a (0xfa bytes) 000362[0] 1779698260.421190 read 0x85a235 thru 0x85c130 (0x1efc bytes) 000365[1] 1779698260.492090 trunc from 0x9e7cab to 0x0e693b (0x90136f bytes) 000366[0] 1779698260.536519 mapwrite 0x5f37c0 thru 0x5f5543 (0x1d84 bytes) 000367[0] 1779698260.713579 trunc from 0x5f5544 to 0x331094 (0x2c44af bytes) 000368[0] 1779698260.744890 read 0x1edd76 thru 0x1eea1d (0xca8 bytes) 000372[0] 1779698260.970690 write 0x6279cb thru 0x62882d (0xe63 bytes) 000373[0] 1779698260.974915 write 0x7d2bcf thru 0x7d414e (0x1580 bytes) 000376[0] 1779698261.027034 write 0x27e9c7 thru 0x27ea8a (0xc4 bytes) 000377[1] 1779698261.028547 mapread 0x6994a5 thru 0x69a418 (0xf74 bytes) 000379[0] 1779698261.370545 trunc from 0x7d414f to 0x4a92c5 (0x32ae89 bytes) 000381[0] 1779698261.494568 trunc from 0x4a92c6 to 0x66f6eb (0x1c6426 bytes) 000383[0] 1779698261.545132 read 0x303948 thru 0x305613 (0x1ccc bytes) 000386[0] 1779698261.747660 read 0x390bd8 thru 0x39208f (0x14b8 bytes) 000387[1] 1779698261.813736 mapread 0x0ec8c4 thru 0x0ee745 (0x1e82 bytes) 000390[0] 1779698262.048878 write 0x01e556 thru 0x01f928 (0x13d3 bytes) 000392[1] 1779698262.201106 mapread 0x00d47d thru 0x00e7a8 (0x132c bytes) 000393[0] 1779698262.279757 write 0x88a861 thru 0x88bcca (0x146a bytes) 000394[0] 1779698262.362931 trunc from 0x88bccb to 0x2fe220 (0x58daaa bytes) 000396[1] 1779698262.418688 trunc from 0x2fe221 to 0x13c6d6 (0x1c1b4a bytes) 000398[1] 1779698262.647269 read 0x0470c1 thru 0x0479e9 (0x929 bytes) 000400[1] 1779698262.729634 write 0x9405cb thru 0x942194 (0x1bca bytes) 000403[0] 1779698262.764300 read 0x91b09a thru 0x91b4b6 (0x41d bytes) 000404[0] 1779698262.823851 trunc from 0x942195 to 0x178342 (0x7c9e52 bytes) 000405[1] 1779698263.035338 read 0x0c2241 thru 0x0c2ba6 (0x966 bytes) 000406[1] 1779698263.208876 mapwrite 0x3d8ef6 thru 0x3da35e (0x1469 bytes) 000407[0] 1779698263.525202 mapread 0x052f06 thru 0x054126 (0x1221 bytes) 000409[0] 1779698263.698517 write 0x2e9a37 thru 0x2ea4e1 (0xaab bytes) 000412[0] 1779698263.875815 mapwrite 0x166e74 thru 0x168b5f (0x1cec bytes) 000413[0] 1779698264.140736 trunc from 0x3da35f to 0x4cf942 (0xf55e4 bytes) 000415[1] 1779698264.299261 write 0x03ab62 thru 0x03b5d2 (0xa71 bytes) 000419[1] 1779698264.450520 trunc from 0x4cf943 to 0x6d323f (0x2038fd bytes) 000424[0] 1779698264.579251 mapwrite 0x912527 thru 0x912d9a (0x874 bytes) 000425[1] 1779698264.826756 write 0x7e3a5d thru 0x7e4ed1 (0x1475 bytes) 000428[0] 1779698265.106172 mapread 0x469ae2 thru 0x46b64d (0x1b6c bytes) 000430[0] 1779698265.199561 read 0x8c53f9 thru 0x8c60b1 (0xcb9 bytes) 000432[0] 1779698265.212500 mapread 0x3de053 thru 0x3df39d (0x134b bytes) 000433[0] 1779698265.240673 write 0x4b370f thru 0x4b4c6d (0x155f bytes) 000434[0] 1779698265.295953 mapread 0x819324 thru 0x81a4ad (0x118a bytes) 000436[1] 1779698265.366403 mapwrite 0x444cb5 thru 0x446684 (0x19d0 bytes) 000441[1] 1779698265.663523 write 0x78844c thru 0x789c6e (0x1823 bytes) 000444[1] 1779698265.803032 write 0x50875e thru 0x5088b5 (0x158 bytes) 000449[1] 1779698266.009947 trunc from 0x912d9b to 0x8886aa (0x8a6f0 bytes) 000451[0] 1779698266.226711 write 0x007691 thru 0x00832c (0xc9c bytes) 000452[1] 1779698266.320811 write 0x4b7ae0 thru 0x4b91f5 (0x1716 bytes) 000453[0] 1779698266.386636 trunc from 0x8886ab to 0x4c09d6 (0x3c7cd4 bytes) 000454[0] 1779698266.553315 mapread 0x4903ac thru 0x491c18 (0x186d bytes) 000455[0] 1779698266.587332 write 0x860e22 thru 0x8623bb (0x159a bytes) 000456[0] 1779698266.597226 trunc from 0x8623bc to 0x340cff (0x5216bc bytes) 000457[0] 1779698266.646801 read 0x0eee18 thru 0x0efc85 (0xe6e bytes) 000458[0] 1779698266.859924 trunc from 0x340d00 to 0x90a797 (0x5c9a98 bytes) 000459[0] 1779698266.930637 write 0x62cd0c thru 0x62de03 (0x10f8 bytes) 000460[1] 1779698267.039129 mapread 0x09af71 thru 0x09c0d8 (0x1168 bytes) 000461[0] 1779698267.154770 read 0x2f4c54 thru 0x2f5f25 (0x12d2 bytes) 000467[0] 1779698267.293258 mapwrite 0x983c6a thru 0x98482e (0xbc5 bytes) 000468[1] 1779698267.442778 write 0x22a7da thru 0x22c19c (0x19c3 bytes) 000469[0] 1779698267.485405 mapread 0x2e5dbb thru 0x2e7c51 (0x1e97 bytes) 000470[1] 1779698267.548331 mapwrite 0x009917 thru 0x00b674 (0x1d5e bytes) 000471[1] 1779698267.691450 read 0x341441 thru 0x342d99 (0x1959 bytes) 000472[1] 1779698267.722137 write 0x410ba3 thru 0x411ecc (0x132a bytes) 000475[1] 1779698267.920940 mapread 0x1ffdb0 thru 0x20158b (0x17dc bytes) 000477[1] 1779698268.010916 mapread 0x201015 thru 0x2015f5 (0x5e1 bytes) 000478[1] 1779698268.049920 read 0x1438fa thru 0x145526 (0x1c2d bytes) 000480[0] 1779698268.106370 read 0x6a66ee thru 0x6a80cb (0x19de bytes) 000482[0] 1779698268.197155 mapread 0x71590d thru 0x715b13 (0x207 bytes) 000484[1] 1779698268.302417 mapwrite 0x9f9d3d thru 0x9fa0e1 (0x3a5 bytes) 000487[1] 1779698268.517968 trunc from 0x9fa0e2 to 0x7fa268 (0x1ffe79 bytes) 000488[0] 1779698268.580118 mapwrite 0x252206 thru 0x2539be (0x17b9 bytes) 000489[1] 1779698268.707349 read 0x1090a3 thru 0x10a781 (0x16df bytes) 000490[1] 1779698268.756391 read 0x2bea29 thru 0x2bf1f8 (0x7d0 bytes) 000494[1] 1779698268.835358 mapwrite 0x606041 thru 0x606600 (0x5c0 bytes) 000497[1] 1779698269.002026 trunc from 0x7fa269 to 0x839a95 (0x3f82d bytes) 000498[0] 1779698269.096042 write 0x667e8e thru 0x6680e5 (0x258 bytes) 000500[1] 1779698269.251507 mapwrite 0x4c0888 thru 0x4c0dbe (0x537 bytes) All operations completed A-OK! PASS 16b (38s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 04:37:57 (1779698277) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (6s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 04:38:02 (1779698282) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.528705 s, 15.9 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.601923 s, 13.9 MB/s ldlm.namespaces.MGC192.168.201.110@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff9ae5833e3800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff9ae584d36800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff9ae5833e3800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff9ae5833e3800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 13.4927 s, 7.8 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.74719 s, 22.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 3.48432 s, 30.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 7.16904 s, 14.6 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.9604 s, 10.5 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.62944 s, 22.7 MB/s PASS 16d (60s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 04:39:02 (1779698342) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.41274 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23012 s, 4.6 MB/s PASS 16e (9s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 04:39:11 (1779698351) TIMEOUT PASS 16f (31s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 04:39:43 (1779698383) TIMEOUT PASS 16g (30s) == sanityn test 16h: mmap read after truncate file ======= 04:40:13 (1779698413) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16h (10s) == sanityn test 16i: read after truncate file ============ 04:40:23 (1779698423) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (8s) == sanityn test 16j: race dio with buffered i/o ========== 04:40:32 (1779698432) 1+0 records in 1+0 records out 26214400 bytes (26 MB, 25 MiB) copied, 3.09353 s, 8.5 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.37657 s, 11.0 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 4.0634 s, 6.5 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.91877 s, 13.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.66789 s, 9.8 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.54497 s, 10.3 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 3.22986 s, 8.1 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.82874 s, 14.3 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.18398 s, 12.0 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 3.83108 s, 6.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 4.23109 s, 6.2 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.5051 s, 10.5 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.72933 s, 9.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 3.33967 s, 7.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 3.452 s, 7.6 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 3.3287 s, 7.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 3.66498 s, 7.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 4.40714 s, 5.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 4.53102 s, 5.8 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (52s) == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 04:41:24 (1779698484) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 Mon May 25 04:42:34 EDT 2026 + fsx 2 done: rc=0 Mon May 25 04:42:35 EDT 2026 + fsx 3 done: rc=0 Mon May 25 04:42:35 EDT 2026 + fsx 4 done: rc=0 Mon May 25 04:42:35 EDT 2026 + fsx 5 done: rc=0 Mon May 25 04:42:35 EDT 2026 PASS 16k (84s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 04:42:49 (1779698569) fail_loc=0x8000030a PASS 17 (14s) == sanityn test 18: mmap sanity check =========================================================================================== 04:43:03 (1779698583) mmap test1: basic mmap operation (PASS, 0.27773s) mmap test2: MAP_PRIVATE not write back (PASS, 0.21375s) mmap test3: concurrent mmap ops on two nodes (PASS, 11.213s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 4.7697s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.26794s) mmap test6: check mmap write/read content on two nodes (PASS, 0.33859s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: multi-thread mmap access (PASS, 2.1878s) mmap test11: multi-thread shared mmap (PASS, 2.4891s) mmap test10: mtime not change for readonly mmap access (PASS, 1.1763s) PASS 18 (34s) == sanityn test 19: test concurrent uncached read races ========================================================================= 04:43:37 (1779698617) oleg110-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg110-client: oleg110-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (6s) == sanityn test 20: test extra readahead page left in cache ============================================================== 04:43:43 (1779698623) PASS 20 (10s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 04:43:53 (1779698633) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (10s) == sanityn test 23: others should see updated atime while another read============================================================== 04:44:03 (1779698643) now is 1779698648 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 new atime is 1779698709 PASS 23 (74s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 04:45:17 (1779698717) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 2176 2206464 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 2176 2206464 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 524.8K 386 524.4K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 400 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 114.5K 402 114.2K 1% /mnt/lustre[OST:1] filesystem_summary: 229.4K 386 229.0K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 524.8K 386 524.4K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 400 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 402 114.2K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.4K 386 229.0K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 2.1M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 8.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 30.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 38.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 537416 386 537030 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118000 400 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117298 402 116896 1% /mnt/lustre2[OST:1] filesystem_summary: 234882 386 234496 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 2176 2206464 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 524.8K 386 524.4K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 400 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 402 114.2K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.4K 386 229.0K 1% /mnt/lustre2 Reader error: 'No devices found' at 0 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 537416 386 537030 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118000 400 117600 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 117298 402 116896 1% /mnt/lustre[OST:1] filesystem_summary: 234882 386 234496 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 537416 386 537030 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118000 400 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117298 402 116896 1% /mnt/lustre2[OST:1] filesystem_summary: 234882 386 234496 1% /mnt/lustre2 Reader error: 'No devices found' at 0 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 2176 2206464 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 2176 2206464 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre2 PASS 24a (11s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 04:45:28 (1779698728) PASS 24b (9s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 04:45:37 (1779698737) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (10s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 04:45:47 (1779698747) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (4s) == sanityn test 26a: allow mtime to get older ============ 04:45:52 (1779698752) utime: good mknod times 1779698752 <= 1779698754 <= 1779698754 for /mnt/lustre/f26a utime: good mknod times 1779698752 <= 1779698754 <= 1779698754 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (9s) == sanityn test 26b: sync mtime between ost and mds ====== 04:46:01 (1779698761) PASS 26b (10s) == sanityn test 26c: set-in-past on open file is not lost on close ========================================================== 04:46:11 (1779698771) multiop /mnt/lustre/f26c.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 File: /mnt/lustre/f26c.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502396 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2026-05-25 04:46:13.000000000 -0400 Birth: 2026-05-25 04:46:14.000000000 -0400 File: /mnt/lustre2/f26c.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502396 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2026-05-25 04:46:13.000000000 -0400 Birth: 2026-05-25 04:46:14.000000000 -0400 PASS 26c (10s) SKIP: sanityn test_27 skipping excluded test 27 == sanityn test 30: recreate file race =================== 04:46:23 (1779698783) 0 PASS 30 (11s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 04:46:34 (1779698794) fail_loc=0x314 PASS 31a (11s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 04:46:45 (1779698805) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg110-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.201.110@tcp:/lustre 7542784 40960 7497728 1% /mnt/lustre2 PASS 31b (29s) == sanityn test 31r: open-rename(replace) race =========== 04:47:15 (1779698835) -rw-r--r-- 1 root root 0 May 25 04:47 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (13s) == sanityn test 31s: open should not revalidate invalid dentry ========================================================== 04:47:27 (1779698847) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 1207: 43141 Terminated tail -f $tfile PASS 31s (11s) == sanityn test 31t: getattr should not revalidate invalid dentry ========================================================== 04:47:38 (1779698858) PASS 31t (11s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 04:47:52 (1779698872) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (6s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 04:47:58 (1779698878) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (5s) == sanityn test 33d: dependent transactions should trigger COS ========================================================== 04:48:03 (1779698883) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (5s) == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 04:48:08 (1779698888) SKIP: sanityn test_33e needs >= 2 MDTs SKIP 33e (4s) == sanityn test 34: no lock timeout under IO ============= 04:48:13 (1779698893) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg110-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9ae5833e3800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff9ae5833e3800.ost_server_uuid in IDLE state after 0 sec oleg110-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9ae5833e3800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff9ae5833e3800.ost_server_uuid in FULL state after 0 sec oleg110-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9ae5833e3800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff9ae5833e3800.ost_server_uuid in IDLE state after 0 sec oleg110-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9ae5833e3800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff9ae5833e3800.ost_server_uuid in FULL state after 0 sec oleg110-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9ae5833e3800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff9ae5833e3800.ost_server_uuid in IDLE state after 0 sec oleg110-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9ae5833e3800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff9ae5833e3800.ost_server_uuid in FULL state after 0 sec PASS 34 (154s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 04:50:47 (1779699047) fail_loc=0x80000317 Race attempt 0 Wait for 49197 49216 for 60 sec... fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.201.110@tcp:/lustre 7.2G 40M 7.2G 1% /mnt/lustre 192.168.201.110@tcp:/lustre 7.2G 40M 7.2G 1% /mnt/lustre2 - open/close 538 (time 1779699067.79 total 10.02 last 53.71) - open/close 1056 (time 1779699077.80 total 20.03 last 51.75) - open/close 1574 (time 1779699087.81 total 30.03 last 51.76) - open/close 2138 (time 1779699097.84 total 40.07 last 56.21) - open/close 2644 (time 1779699107.85 total 50.07 last 50.57) - open/close 3189 (time 1779699117.87 total 60.10 last 54.37) - open/close 3511 (time 1779699127.89 total 70.12 last 32.12) - open/close 3914 (time 1779699137.91 total 80.14 last 40.22) total: 4000 open/close in 82.06 seconds: 48.74 ops/second PASS 35 (92s) == sanityn test 36: handle ESTALE/open-unlink correctly == 04:52:20 (1779699140) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg110-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.4406 s, 36.4 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7497728) after_dd(7439360) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.13485 s, 46.2 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 sleep 5 for ZFS MDS Waiting for MDT destroys to complete *** cycle(1) *** before(7497728) after_dd(7439360) after(7490560) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.03503 s, 50.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7490560) after_dd(7439360) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.948464 s, 55.3 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7497728) after_dd(7440384) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.24629 s, 42.1 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7497728) after_dd(7439360) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.15068 s, 45.6 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7497728) after_dd(7440384) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.00624 s, 52.1 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7497728) after_dd(7439360) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.72932 s, 30.3 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7497728) after_dd(7439360) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.23975 s, 42.3 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7497728) after_dd(7439360) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.14919 s, 45.6 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7497728) after_dd(7434240) after(7497728) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.22601 s, 42.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7497728) after_dd(7439360) after(7497728) PASS 36 (407s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 04:59:07 (1779699547) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 - create 857 (time 1779699564.10 total 10.01 last 85.60) - create 1717 (time 1779699574.10 total 20.01 last 86.00) - create 2521 (time 1779699584.10 total 30.02 last 80.33) - create 3442 (time 1779699594.11 total 40.02 last 92.08) - create 4253 (time 1779699604.12 total 50.03 last 81.02) - create 5099 (time 1779699614.12 total 60.04 last 84.55) - create 5919 (time 1779699624.14 total 70.05 last 81.88) - create 6629 (time 1779699634.14 total 80.06 last 70.99) - create 7251 (time 1779699644.14 total 90.06 last 62.20) - create 7942 (time 1779699654.15 total 100.06 last 69.04) - create 8715 (time 1779699664.16 total 110.08 last 77.19) - create 9560 (time 1779699674.17 total 120.09 last 84.44) total: 10000 create in 125.04 seconds: 79.97 ops/second PASS 37 (144s) == sanityn test 39a: file mtime does not change after rename ========================================================== 05:01:31 (1779699691) repeat after cancel_lru_locks PASS 39a (12s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 05:01:43 (1779699703) repeat after cancel_lru_locks PASS 39b (14s) == sanityn test 39c: check truncate mtime update ================================================================================ 05:01:57 (1779699717) repeat after cancel_lru_locks PASS 39c (11s) == sanityn test 39d: sync write should update mtime ====== 05:02:09 (1779699729) fail_loc=0x411 fail_loc=0 PASS 39d (11s) SKIP: sanityn test_40a skipping ALWAYS excluded test 40a == sanityn test 40b: pdirops: open|create and others ======================================================================== 05:02:22 (1779699742) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (26s) == sanityn test 40c: pdirops: link and others ======================================================================== 05:02:48 (1779699768) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (26s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 05:03:14 (1779699794) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (23s) == sanityn test 40e: pdirops: rename and others ======================================================================== 05:03:38 (1779699818) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (24s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 05:04:01 (1779699841) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (22s) == sanityn test 41b: pdirops: create vs create ======================================================================== 05:04:23 (1779699863) open: File exists Conflict PASS 41b (19s) == sanityn test 41c: pdirops: create vs link ======================================================================== 05:04:42 (1779699882) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (20s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 05:05:02 (1779699902) Conflict PASS 41d (21s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 05:05:23 (1779699923) Conflict PASS 41e (20s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 05:05:43 (1779699943) Conflict PASS 41f (20s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 05:06:03 (1779699963) Conflict PASS 41g (19s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 05:06:22 (1779699982) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 2318: lctl set_param -n fail_loc=0 2>/dev/null || true: No such file or directory Conflict PASS 41h (19s) == sanityn test 41i: reint_open: create vs create ======== 05:06:41 (1779700001) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (1318s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 05:28:40 (1779701320) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (19s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 05:28:58 (1779701338) open: File exists Conflict PASS 42b (18s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 05:29:16 (1779701356) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (18s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 05:29:34 (1779701374) Conflict PASS 42d (19s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 05:29:53 (1779701393) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (19s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 05:30:12 (1779701412) Conflict PASS 42f (20s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 05:30:33 (1779701433) Conflict PASS 42g (18s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 05:30:51 (1779701451) Conflict PASS 42h (18s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 05:31:10 (1779701470) PASS 43a (101s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 05:32:50 (1779701570) Conflict PASS 43b (21s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 05:33:12 (1779701592) Conflict PASS 43c (19s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 05:33:31 (1779701611) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (18s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 05:33:50 (1779701630) Conflict PASS 43e (17s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 05:34:07 (1779701647) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (21s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 05:34:28 (1779701668) stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (21s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 05:34:49 (1779701689) Conflict PASS 43h (24s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 05:35:13 (1779701713) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (5s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 05:35:18 (1779701718) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (173s) == sanityn test 43k: unlink vs create ==================== 05:38:11 (1779701891) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (1769s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 06:07:40 (1779703660) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Mon May 25 06:07:51 EDT 2026 PASS 44a (19s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 06:07:59 (1779703679) open: File exists Conflict PASS 44b (19s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 06:08:18 (1779703698) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (19s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 06:08:37 (1779703717) Conflict PASS 44d (19s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 06:08:56 (1779703736) Conflict PASS 44e (19s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 06:09:15 (1779703755) Conflict PASS 44f (19s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 06:09:34 (1779703774) Conflict PASS 44g (19s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 06:09:53 (1779703793) Conflict PASS 44h (19s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 06:10:12 (1779703812) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (5s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 06:10:17 (1779703817) PASS 45a (167s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 06:13:05 (1779703985) Conflict PASS 45b (21s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 06:13:25 (1779704005) Conflict PASS 45c (20s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 06:13:45 (1779704025) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (24s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 06:14:09 (1779704049) Conflict PASS 45e (23s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 06:14:32 (1779704072) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (20s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 06:14:52 (1779704092) stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (21s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 06:15:13 (1779704113) Conflict PASS 45h (24s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 06:15:37 (1779704137) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (6s) == sanityn test 45j: read vs rename ====================== 06:15:43 (1779704143) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (1754s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 06:44:57 (1779705897) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (19s) == sanityn test 46b: pdirops: link vs create ======================================================================== 06:45:17 (1779705917) open: File exists Conflict PASS 46b (22s) == sanityn test 46c: pdirops: link vs link ======================================================================== 06:45:39 (1779705939) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (20s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 06:45:59 (1779705959) Conflict PASS 46d (19s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 06:46:19 (1779705979) Conflict PASS 46e (18s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 06:46:36 (1779705996) Conflict PASS 46f (19s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 06:46:55 (1779706015) Conflict PASS 46g (20s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 06:47:15 (1779706035) Conflict PASS 46h (19s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 06:47:35 (1779706055) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (6s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 06:47:40 (1779706060) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (4s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 06:47:45 (1779706065) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (4s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 06:47:49 (1779706069) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (5s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 06:47:54 (1779706074) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (5s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 06:47:59 (1779706079) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (5s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 06:48:04 (1779706084) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (5s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 06:48:09 (1779706089) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (5s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 06:48:15 (1779706095) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.156644 s, 65.4 kB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (17s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 06:48:31 (1779706111) 0+1 records in 0+1 records out 158 bytes copied, 0.0276385 s, 5.7 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3581: kill: (217995) - No such process PASS 51a (13s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 06:48:45 (1779706125) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0204824 s, 50.0 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162194 s, 6.5 MB/s 3145728 PASS 51b (25s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 06:49:09 (1779706149) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0733992 s, 14.0 kB/s PASS 51c (17s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 06:49:26 (1779706166) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0908581 s, 11.5 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (13s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 06:49:39 (1779706179) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2452 0x994 0x280000400 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2452 0x994 0x280000400 read lease(1) has applied. PASS 51e (13s) == sanityn test 54: rename locking ======================= 06:49:52 (1779706192) llite.lustre-ffff9ae5833e3800.xattr_cache=1 llite.lustre-ffff9ae584d36800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505009 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-25 06:49:55.000000000 -0400 Modify: 2026-05-25 06:49:55.000000000 -0400 Change: 2026-05-25 06:49:55.000000000 -0400 Birth: 2026-05-25 06:49:56.000000000 -0400 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 7680 Blocks: 15 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505012 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-25 06:50:02.000000000 -0400 Modify: 2026-05-25 06:50:04.000000000 -0400 Change: 2026-05-25 06:50:04.000000000 -0400 Birth: 2026-05-25 06:50:03.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 7680 Blocks: 15 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505015 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-25 06:50:10.000000000 -0400 Modify: 2026-05-25 06:50:10.000000000 -0400 Change: 2026-05-25 06:50:10.000000000 -0400 Birth: 2026-05-25 06:50:11.000000000 -0400 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 7680 Blocks: 15 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505014 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-25 06:50:10.000000000 -0400 Modify: 2026-05-25 06:50:11.000000000 -0400 Change: 2026-05-25 06:50:11.000000000 -0400 Birth: 2026-05-25 06:50:11.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1/d2 Size: 7680 Blocks: 15 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505018 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-25 06:50:17.000000000 -0400 Modify: 2026-05-25 06:50:19.000000000 -0400 Change: 2026-05-25 06:50:19.000000000 -0400 Birth: 2026-05-25 06:50:18.000000000 -0400 File: /mnt/lustre/d1 Size: 7680 Blocks: 15 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505017 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-25 06:50:17.000000000 -0400 Modify: 2026-05-25 06:50:19.000000000 -0400 Change: 2026-05-25 06:50:19.000000000 -0400 Birth: 2026-05-25 06:50:18.000000000 -0400 PASS 54 (43s) == sanityn test 55a: rename vs unlink target dir ========= 06:50:35 (1779706235) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (19s) == sanityn test 55b: rename vs unlink source dir ========= 06:50:54 (1779706254) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (20s) == sanityn test 55c: rename vs unlink orphan target dir == 06:51:14 (1779706274) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (24s) == sanityn test 55d: rename file vs link ================= 06:51:39 (1779706299) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (19s) == sanityn test 55e: rename race AB/BA under the same parent dir ========================================================== 06:51:58 (1779706318) SKIP: sanityn test_55e needs >= 2 MDTs SKIP 55e (5s) == sanityn test 55f: rename: (P1/A -> P2/B) race with (P2/B -> P1/A) ========================================================== 06:52:04 (1779706324) fail_loc=0x155 We survived after P1A_P2B/P2B_P1A race test fail_loc=0x0 PASS 55f (25s) == sanityn test 55g: rename: race with trylock =========== 06:52:28 (1779706348) mdt.lustre-MDT0000.enable_rename_trylock=1 fail_loc=0x155 mv: cannot move '/mnt/lustre2/d55g.sanityn/b1/b2' to a subdirectory of itself, '/mnt/lustre2/d55g.sanityn/a1/a2/a3/a4/b2' We survived race with trylock test fail_loc=0x0 PASS 55g (28s) == sanityn test 56a: test llverdev with single large stripe ========================================================== 06:52:56 (1779706376) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1779706379 chunksize: 4096 size: 16777216000000 write offset: 16384000000kB 0.401573 MB/s read offset: 16384000000kB inf MB/s llverdev: data verified successfully PASS 56a (165s) == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 06:55:42 (1779706542) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1779706550 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 0.693604 MB/s read offset: 32768000000kB inf MB/s llverdev: data verified successfully PASS 56b (197s) == sanityn test 60: Verify data_version behaviour ======== 06:58:59 (1779706739) PASS 60 (13s) == sanityn test 70a: cd directory && rm directory ======== 06:59:13 (1779706753) PASS 70a (13s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 06:59:26 (1779706766) PASS 70b (15s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 06:59:41 (1779706781) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0109182 s, 6.0 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (6s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 06:59:47 (1779706787) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0268516 s, 2.4 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (6s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 06:59:53 (1779706793) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (6s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 06:59:59 (1779706799) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0250478 s, 2.6 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (4s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 07:00:04 (1779706804) llite.lustre-ffff9ae5833e3800.xattr_cache=1 llite.lustre-ffff9ae584d36800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (10s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 07:00:14 (1779706814) llite.lustre-ffff9ae5833e3800.xattr_cache=1 llite.lustre-ffff9ae584d36800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (11s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 07:00:25 (1779706825) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0294678 s, 34.7 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0278125 s, 36.8 kB/s 230834: get lock2 230838: get lock1 230838: done 230838: get lock2 230834: done 230834: get lock1 fcntl cmd 7 failed: Resource deadlock avoided 230834: cannot get lock1: Resource deadlock avoided 230838: done 230838: exit rc=0 230834: done 230834: exit rc=0 PASS 74 (14s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 07:00:39 (1779706839) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0589694 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0399809 s, 26.2 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4264: 231419 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (15s) == sanityn test 76: Verify MDT open_files listing ======== 07:00:55 (1779706855) opening files: ........ closing files: ........ PASS 76 (102s) == sanityn test 77a: check FIFO NRS policy =============== 07:02:36 (1779706956) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.415207 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0737535 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108752 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147111 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0704994 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122415 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0975997 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0487559 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135969 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136807 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248971 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0625482 s, 16.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.280775 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.322742 s, 3.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206154 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.266603 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.331183 s, 3.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.086789 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0463159 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.048117 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0936134 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0470276 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14573 s, 7.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0647041 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0626588 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0683331 s, 15.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0715081 s, 14.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1231 s, 8.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0825185 s, 12.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160571 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.080391 s, 13.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0888171 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0947457 s, 11.1 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (14s) == sanityn test 77b: check CRR-N NRS policy ============== 07:02:50 (1779706970) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.339554 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0792503 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113157 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147191 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232191 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230148 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203665 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264807 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.281879 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.352833 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.320439 s, 3.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.365417 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.349489 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.371472 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.373139 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.406795 s, 2.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.413499 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.277924 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160655 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163899 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143231 s, 7.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143047 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175394 s, 6.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176652 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145138 s, 7.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188792 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10362 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0841479 s, 12.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161145 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203522 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.21469 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213622 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232891 s, 4.5 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.383573 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0733496 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0831144 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356558 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0553008 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0737938 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162256 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218755 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0742705 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118607 s, 8.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100915 s, 10.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253937 s, 4.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17324 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0355574 s, 29.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122541 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140364 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0995761 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317953 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247132 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329811 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0404285 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100512 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0966132 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794301 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0713747 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.07861 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0490678 s, 21.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0567721 s, 18.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0718426 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144015 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155776 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165886 s, 63.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166522 s, 63.0 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (18s) == sanityn test 77c: check ORR NRS policy ================ 07:03:08 (1779706988) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.560701 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0349181 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.049884 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108254 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152183 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0697483 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10498 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0997369 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0517127 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218218 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.089442 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.284691 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227173 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187573 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146315 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.214812 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253534 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0523868 s, 20.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0509713 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0400249 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152794 s, 68.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.057537 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0678412 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114486 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0723136 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123632 s, 84.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117478 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0608344 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0835745 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0811029 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0460642 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0723219 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0910007 s, 11.5 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.363591 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0366882 s, 28.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0978582 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0677146 s, 15.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0627008 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0565148 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.214737 s, 4.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.272958 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0828406 s, 12.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157141 s, 6.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192632 s, 5.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251448 s, 4.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155787 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144452 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215928 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147743 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17751 s, 5.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=2 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102493 s, 10.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0510388 s, 20.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0867762 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0946614 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.061961 s, 16.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117416 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114353 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.092999 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.097015 s, 10.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150298 s, 7.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260175 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178901 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157683 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15663 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116568 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162122 s, 6.5 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.305946 s, 54.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.050409 s, 20.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0600466 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0674119 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0447171 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0540626 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0506952 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.067431 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163343 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0463079 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114996 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23462 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.233478 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228036 s, 4.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14388 s, 7.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155184 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128558 s, 8.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0949465 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0542999 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134289 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0580089 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0963898 s, 10.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0582643 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0604107 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0929209 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944525 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0964944 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138018 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149608 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147206 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0375873 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115168 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112391 s, 9.3 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (22s) == sanityn test 77d: check TRR nrs policy ================ 07:03:30 (1779707010) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.457976 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242002 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0465356 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308615 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0632779 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0775388 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144296 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118735 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0942743 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0613816 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10343 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0960708 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.287131 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140284 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.363635 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217986 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.454521 s, 2.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0483158 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221979 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0883093 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149292 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0350276 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0670031 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0442402 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0448754 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127882 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121273 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574154 s, 183 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135507 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0513643 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0495223 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13481 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0601877 s, 17.4 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.479721 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0704729 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0920768 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0509572 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108039 s, 9.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140827 s, 7.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103873 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156192 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239409 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18651 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239188 s, 4.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229933 s, 4.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.280158 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.392677 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.413218 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.432696 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.574623 s, 1.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230317 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0931139 s, 11.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132487 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0354187 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.07806 s, 13.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249766 s, 42.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113526 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0476331 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0641785 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021998 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131928 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0976179 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113313 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0385418 s, 27.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0508802 s, 20.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145092 s, 72.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=1 removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.326441 s, 51.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0887206 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0956376 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137469 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171275 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203671 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175458 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161894 s, 6.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200947 s, 5.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.267648 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.296454 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223166 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232231 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.344798 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260193 s, 4.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273553 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.312022 s, 3.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178266 s, 58.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0347193 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299741 s, 35.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260335 s, 40.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159207 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124461 s, 8.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260103 s, 40.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0542194 s, 19.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0483872 s, 21.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136199 s, 7.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029837 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0896176 s, 11.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118757 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0363599 s, 28.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662592 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257879 s, 40.7 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (21s) == sanityn test 77e: check TBF NID nrs policy ============ 07:03:51 (1779707031) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.201.10@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.323971 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0511579 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0578046 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0409072 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152067 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.210024 s, 5.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0961822 s, 10.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.210208 s, 5.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222156 s, 4.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108852 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11821 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119492 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157498 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.252725 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.268797 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13718 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273667 s, 3.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0946114 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209071 s, 50.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0898574 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.067816 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0701019 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.097684 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148409 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108924 s, 9.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115494 s, 9.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0899088 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173525 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205171 s, 51.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247078 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0769972 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0416391 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.093784 s, 11.2 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.475464 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130545 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195299 s, 5.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125081 s, 8.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.081309 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251512 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218035 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230722 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.261352 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.331957 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273884 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.346188 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.305637 s, 3.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.351492 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.449545 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.432587 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.476529 s, 2.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00997213 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109528 s, 95.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00951889 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140127 s, 74.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137717 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324491 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151063 s, 69.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154932 s, 67.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0447338 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0559156 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.087471 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14354 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105202 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112166 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184863 s, 56.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0797133 s, 13.2 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.315649 s, 53.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0404608 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0824605 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0926357 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111936 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184979 s, 56.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110425 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163107 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.085884 s, 12.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0455087 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0655031 s, 16.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159136 s, 6.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171042 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264972 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.365723 s, 2.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.439847 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.508305 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125144 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00955361 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136788 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134812 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195128 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134379 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0927465 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105384 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200925 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134196 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122673 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0800915 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128724 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156463 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145169 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122146 s, 8.6 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.373944 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0394382 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.063161 s, 16.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0474551 s, 22.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0742264 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0820331 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0453134 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0802685 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0378101 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0595876 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216484 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224243 s, 4.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.252404 s, 4.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.202858 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264685 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148925 s, 7.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.309974 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0838732 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0776102 s, 13.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0681555 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0745863 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.085503 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0986646 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105701 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156805 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139803 s, 7.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133539 s, 7.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132393 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131115 s, 8.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20705 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146316 s, 7.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134327 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165477 s, 6.3 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (35s) == sanityn test 77f: check TBF JobID nrs policy ========== 07:04:27 (1779707067) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.361124 s, 46.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0771498 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111376 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.063764 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128919 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185972 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323097 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251308 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0884644 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170374 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.298915 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211969 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.280521 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0471465 s, 22.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0696318 s, 15.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0775106 s, 13.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.300415 s, 3.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0968916 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12683 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170969 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152638 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0523505 s, 20.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138198 s, 7.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106613 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103127 s, 10.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124077 s, 8.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109014 s, 9.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111899 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201398 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118389 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134327 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111427 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121974 s, 8.6 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.367011 s, 45.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0980858 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0970817 s, 10.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0634217 s, 16.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.077073 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112832 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137446 s, 7.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16697 s, 6.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176886 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165038 s, 6.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133246 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199098 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244823 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140993 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158625 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206351 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.284783 s, 3.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0988322 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104044 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0923212 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0545991 s, 19.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0864756 s, 12.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0828962 s, 12.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0941192 s, 11.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00627056 s, 167 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136175 s, 7.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132631 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0949939 s, 11.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10834 s, 9.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148271 s, 70.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121667 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118771 s, 8.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330537 s, 31.7 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.417792 s, 40.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.079743 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.05631 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0457183 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0874948 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273416 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0426482 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145873 s, 7.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10317 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128542 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163491 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239301 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211695 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.256179 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251559 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.285707 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11929 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0401025 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0515669 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0539455 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00918612 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125139 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.332568 s, 3.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.047622 s, 22.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0848677 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109016 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0522616 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0936239 s, 11.2 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158292 s, 6.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127009 s, 82.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0362212 s, 28.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101886 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00897823 s, 117 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.369297 s, 45.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0399558 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860302 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140506 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0647097 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0601121 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156512 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.225448 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0781199 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0575562 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.301321 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.336986 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206009 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.254863 s, 4.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.29134 s, 3.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.311575 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.363196 s, 2.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0832675 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117929 s, 88.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356674 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0788543 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101288 s, 10.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.043087 s, 24.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.080461 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170895 s, 61.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144774 s, 72.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0870064 s, 12.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=4 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107063 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00519233 s, 202 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.073867 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0440643 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00379056 s, 277 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179456 s, 58.4 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (33s) == sanityn test 77g: Change TBF type directly ============ 07:04:59 (1779707099) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (16s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 07:05:15 (1779707115) oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg110-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg110-client: oleg110-server: ssh exited with exit code 2 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Invalid argument oleg110-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Invalid argument oleg110-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Invalid argument oleg110-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 PASS 77h (17s) == sanityn test 77i: Change rank of TBF rule ============= 07:05:32 (1779707132) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (32s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 07:06:04 (1779707164) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.799479 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0855695 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147695 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130132 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164226 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.304332 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.308685 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317241 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.349063 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.27905 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248069 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.372824 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217163 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.335359 s, 3.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.353932 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.266089 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.294832 s, 3.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0698272 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0950286 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0889199 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100901 s, 10.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0994542 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122901 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0829353 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0957308 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105957 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138978 s, 7.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154155 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12324 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116731 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129842 s, 8.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161063 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.221237 s, 4.7 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.1404 s, 10.3 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.643 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (74s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 07:07:18 (1779707238) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg110-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg110-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg110-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg110-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.32997 s, 12.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0902077 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159937 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117051 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191127 s, 5.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123972 s, 8.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0518406 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.300566 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.243984 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220049 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23554 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.299834 s, 3.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.3451 s, 3.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216394 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253962 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.349099 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.40793 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0623676 s, 16.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0855951 s, 12.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0673193 s, 15.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0945723 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101224 s, 10.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0733561 s, 14.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105638 s, 9.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.071382 s, 14.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106598 s, 9.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0741941 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0820752 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0750313 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0911793 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100979 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103566 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109609 s, 9.6 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.0917 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9232 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg110-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg110-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg110-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg110-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.33029 s, 12.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0958346 s, 10.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0929617 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0673042 s, 15.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143695 s, 7.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132438 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159867 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177356 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0400875 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16068 s, 6.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237054 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.092337 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109885 s, 9.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215944 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.299473 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182221 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0608179 s, 17.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178472 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138896 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147628 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107372 s, 9.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138907 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217436 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131625 s, 8.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.24591 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241301 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.246758 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.225908 s, 4.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180564 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186461 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217362 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173718 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245019 s, 4.3 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.5675 s, 5.1 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.7209 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (170s) == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 07:10:09 (1779707409) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg110-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg110-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg110-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg110-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.36239 s, 12.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10301 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159173 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141064 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104909 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175713 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.242687 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.072456 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.235146 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0816292 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.325602 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192947 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.337679 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.398736 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23223 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196399 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.387028 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14811 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153882 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136847 s, 7.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147855 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176861 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188221 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171754 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157924 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191145 s, 5.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162424 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16604 s, 6.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185971 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169467 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168993 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182512 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187979 s, 5.6 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.2089 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.0115 s, 5.2 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg110-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg110-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg110-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg110-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.339 s, 12.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0735417 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0881226 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0936267 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170119 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157174 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.204716 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.275961 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.27015 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.231522 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.292692 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.331569 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.461783 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.427353 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.482313 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.524297 s, 2.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.546187 s, 1.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102555 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142717 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0983657 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0749016 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118525 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101509 s, 10.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105555 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0969755 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157924 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156505 s, 6.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112957 s, 9.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108013 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108344 s, 9.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108457 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.202593 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108638 s, 9.7 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.9594 s, 5.0 MB/s Write runtime is 22 s, speed is 4.545454 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.0167 s, 5.2 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (170s) == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 07:12:58 (1779707578) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.434593 s, 38.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0464464 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.056 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101657 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206684 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112499 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109563 s, 9.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18389 s, 5.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.231355 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240653 s, 4.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.311008 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.322646 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232772 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232862 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.236174 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237706 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.34185 s, 3.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313658 s, 33.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.063226 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0350587 s, 29.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315582 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.071173 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0409536 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111313 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0810773 s, 12.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0732095 s, 14.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0645899 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0793761 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0900502 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0943367 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0769515 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0853154 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0390379 s, 26.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.7026 s, 9.8 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.0746 s, 10.4 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.10@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.10@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.902717 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0606858 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0536704 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0939634 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127735 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0535418 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126375 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187461 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101737 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138778 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.259995 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273645 s, 3.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.300594 s, 3.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220253 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211309 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196954 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228367 s, 4.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175822 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193573 s, 5.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157151 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.297991 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208642 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193703 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197631 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.202296 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237091 s, 4.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197039 s, 5.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.234199 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.258612 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239375 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.247414 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255449 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.269006 s, 3.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=1 removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 11.9833 s, 8.8 MB/s Write runtime is 13 s, speed is 7.692307 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.1762 s, 10.3 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.10@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.445208 s, 37.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.062164 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0803471 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0412338 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0528728 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0572519 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0703624 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0477467 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0999061 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145649 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186278 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0617699 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103782 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.202824 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.296033 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.310054 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17307 s, 6.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0697461 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0353424 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0901314 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0547165 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0715204 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0789112 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0814713 s, 12.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143282 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0933799 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0802153 s, 13.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0756053 s, 13.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108291 s, 9.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0989977 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168903 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101017 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134083 s, 7.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.90394 s, 11.8 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.55322 s, 18.9 MB/s Read runtime is 7 s, speed is 14.285714 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.539151 s, 31.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0705886 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111126 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0865753 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0912464 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124817 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115975 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0755624 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137477 s, 7.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.226163 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191541 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182193 s, 5.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227475 s, 4.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215364 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200749 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.274119 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.321817 s, 3.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155348 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141463 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156561 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0854474 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0906178 s, 11.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0868855 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170038 s, 6.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15144 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100881 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114611 s, 9.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164384 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108349 s, 9.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109632 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192223 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133352 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138254 s, 7.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 12.5573 s, 8.4 MB/s Write runtime is 13 s, speed is 7.692307 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.1103 s, 10.4 MB/s Read runtime is 12 s, speed is 8.333333 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 13.4173 s, 7.8 MB/s Write runtime is 15 s, speed is 6.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.2333 s, 10.2 MB/s Read runtime is 12 s, speed is 8.333333 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.03505 s, 16.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0763711 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100902 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124931 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13058 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120225 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154424 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0945284 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0359039 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0525134 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149076 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.291641 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121589 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0753463 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.365769 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.397762 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206823 s, 5.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114822 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106393 s, 9.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174703 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125881 s, 8.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211131 s, 5.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14568 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0953159 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132316 s, 7.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134915 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143837 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138636 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144821 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139851 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128856 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174659 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179216 s, 5.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.3693 s, 5.1 MB/s Write runtime is 22 s, speed is 4.545454 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.7136 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.806358 s, 20.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0917987 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0918986 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0648457 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0514403 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0584798 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0848626 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.06552 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0970803 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0659358 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0907973 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164767 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201555 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224477 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317702 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162243 s, 6.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.281836 s, 3.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0393262 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0688498 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104348 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142663 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143398 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0857727 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0699695 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105109 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14386 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.091615 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117413 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126357 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117207 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127521 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127089 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155171 s, 6.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 14.5531 s, 7.2 MB/s Write runtime is 16 s, speed is 6.250000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.0575 s, 10.4 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.664652 s, 25.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0875541 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0819757 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0553014 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.249325 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.25549 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218119 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165214 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11922 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121046 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0685205 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216996 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192639 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0527188 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215409 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.246706 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0996167 s, 10.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0615992 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10863 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0725186 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0334437 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111623 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140998 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136448 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166613 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116868 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.090867 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136894 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013971 s, 75.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0473042 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324174 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140969 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0349937 s, 30.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 13.2679 s, 7.9 MB/s Write runtime is 15 s, speed is 6.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.2081 s, 10.3 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 13.7705 s, 7.6 MB/s Write runtime is 15 s, speed is 6.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.1435 s, 10.3 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (503s) == sanityn test 77kb: Check different granular TBF type combination: jobid+opcode ========================================================== 07:21:21 (1779708081) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.760033 s, 22.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0736522 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278843 s, 37.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0791118 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.274605 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.282212 s, 3.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232301 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.27245 s, 3.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.236593 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240979 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253534 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.405543 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.355482 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.382263 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.414714 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.396389 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.453163 s, 2.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0438882 s, 23.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113343 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0730371 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106411 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.210779 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110442 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132997 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136068 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241165 s, 4.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230465 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154241 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167521 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161805 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163825 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18929 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.202115 s, 5.2 MB/s removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77kb.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 11.9339 s, 8.8 MB/s Write runtime is 14 s, speed is 7.142857 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.0043 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kb (70s) == sanityn test 77kc: Check different granular TBF type combination: nid+opcode ========================================================== 07:22:31 (1779708151) ost.OSS.ost_io.nrs_policies=tbf nid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.10@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.10@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.655289 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0629516 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0732978 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102872 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168565 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184905 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205668 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18655 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181665 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196436 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245584 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240533 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.277689 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248547 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.286286 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.297188 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.300788 s, 3.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196984 s, 5.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167846 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218053 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188173 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160926 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161349 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160551 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16536 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172018 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182903 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222912 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.210322 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208813 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216061 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.226113 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251854 s, 4.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=2 removed '/mnt/lustre/d77kc.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77kc.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77kc.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 11.4287 s, 9.2 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.0366 s, 10.4 MB/s Read runtime is 12 s, speed is 8.333333 IOPS cleanup_tbf /mnt/lustre/d77kc.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kc (63s) == sanityn test 77kd: Check different granular TBF type combination: nid+jobid ========================================================== 07:23:34 (1779708214) ost.OSS.ost_io.nrs_policies=tbf nid+jobid ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.10@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kd.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.51482 s, 32.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0457174 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0466349 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.246628 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161502 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313294 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0835671 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237273 s, 44.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.305172 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.323349 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0722813 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284588 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341269 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139592 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160819 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0466534 s, 22.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132601 s, 7.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122256 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0524309 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0997127 s, 10.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0497707 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0756877 s, 13.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116611 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128026 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162027 s, 64.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12218 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109708 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00588911 s, 178 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164017 s, 63.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020813 s, 50.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320159 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121177 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229706 s, 45.6 MB/s removed '/mnt/lustre/d77kd.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77kd.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77kd.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kd.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.3333 s, 10.1 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kd.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.82495 s, 18.0 MB/s Read runtime is 7 s, speed is 14.285714 IOPS cleanup_tbf /mnt/lustre/d77kd.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kd (56s) == sanityn test 77ke: Check different granular TBF type combination: jobid+opcode ========================================================== 07:24:30 (1779708270) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ke.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.720229 s, 23.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0793799 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0538286 s, 19.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150614 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124899 s, 8.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166057 s, 6.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160628 s, 6.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183137 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105942 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201135 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206471 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16566 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.259018 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.265086 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245849 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.285327 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.285068 s, 3.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186295 s, 5.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172897 s, 6.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0835901 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192054 s, 5.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101405 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12687 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11232 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185486 s, 5.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134763 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136892 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.226734 s, 4.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144633 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147355 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166961 s, 6.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165884 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191528 s, 5.5 MB/s removed '/mnt/lustre/d77ke.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77ke.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77ke.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ke.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 13.0034 s, 8.1 MB/s Write runtime is 14 s, speed is 7.142857 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ke.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.2972 s, 10.2 MB/s Read runtime is 12 s, speed is 8.333333 IOPS cleanup_tbf /mnt/lustre/d77ke.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.9883 s, 9.5 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.0388 s, 10.4 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77ke.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77ke (115s) == sanityn test 77kf: Check different granular TBF type combination: uid+gid ========================================================== 07:26:25 (1779708385) ost.OSS.ost_io.nrs_policies=tbf uid+gid ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kf.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.27853 s, 13.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171152 s, 61.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0377525 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0418902 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113991 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120706 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0811482 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11387 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0536432 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0395614 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0536083 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0863582 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105458 s, 9.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.294073 s, 3.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215601 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224462 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218352 s, 4.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125679 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0383541 s, 27.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0454642 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174031 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012961 s, 80.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117899 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12211 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105544 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181152 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139916 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136085 s, 7.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312989 s, 33.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167473 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147359 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173287 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191019 s, 5.5 MB/s removed '/mnt/lustre/d77kf.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77kf.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77kf.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kf.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.2598 s, 5.2 MB/s Write runtime is 22 s, speed is 4.545454 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kf.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.977 s, 5.2 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77kf.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kf (83s) == sanityn test 77kg: check different granular TBF type combination: uid+opcode ========================================================== 07:27:48 (1779708468) ost.OSS.ost_io.nrs_policies=tbf uid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.932495 s, 18.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0789108 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0706549 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0580656 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20628 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.249093 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.324002 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.311151 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.266683 s, 3.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=2 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.301711 s, 3.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.387866 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.297804 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.379305 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.379503 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.377384 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.397621 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.413152 s, 2.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0807697 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100754 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304265 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327929 s, 32.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126937 s, 8.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0622829 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0611848 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.065236 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137457 s, 7.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113599 s, 9.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161484 s, 64.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0969462 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125262 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121289 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122517 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178297 s, 5.9 MB/s removed '/mnt/lustre/d77kg.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77kg.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77kg.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 16.3122 s, 6.4 MB/s Write runtime is 18 s, speed is 5.555555 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.1369 s, 10.3 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/nrs_r_oleg110-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.73102 s, 23.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.068346 s, 15.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0875139 s, 12.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.242967 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191035 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16799 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183351 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.212359 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.383369 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.383263 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.404676 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.437422 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.404416 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.313239 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.477959 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.475461 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.38653 s, 2.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=2 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.036907 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137183 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106553 s, 98.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0487455 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0984962 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0678738 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174842 s, 60.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021032 s, 49.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0727303 s, 14.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102931 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178082 s, 58.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161629 s, 64.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163279 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141024 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119966 s, 87.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171474 s, 6.1 MB/s removed '/mnt/lustre/d77kg.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77kg.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77kg.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 15.3411 s, 6.8 MB/s Write runtime is 16 s, speed is 6.250000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.0191 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 14.5854 s, 7.2 MB/s Write runtime is 15 s, speed is 6.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.0489 s, 10.4 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kg (173s) == sanityn test 77kh: Verify that Project ID support for NRS TBF rule ========================================================== 07:30:41 (1779708641) using SAVE_PROJECT_SUPPORTED=0 Stopping clients: oleg110-client.virtnet /mnt/lustre (opts:) Stopping client oleg110-client.virtnet /mnt/lustre opts: Stopping clients: oleg110-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg110-client.virtnet /mnt/lustre2 opts: Stopping /mnt/lustre-mds1 (opts:-f) on oleg110-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg110-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg110-server sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime) proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755) securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev) devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000) tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755) tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755) cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd) pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime) bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700) cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio) cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,rdma) cgroup on /sys/fs/cgroup/misc type cgroup (rw,nosuid,nodev,noexec,relatime,misc) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event) none on /sys/kernel/tracing type tracefs (rw,relatime) configfs on /sys/kernel/config type configfs (rw,relatime) /dev/nbd0 on / type ext4 (ro,relatime,stripe=32) rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) debugfs on /sys/kernel/debug type debugfs (rw,relatime) hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M) mqueue on /dev/mqueue type mqueue (rw,relatime) none on /mnt type ramfs (rw,relatime) /dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime) none on /var/lib/stateless/writable type tmpfs (rw,relatime) none on /var/cache/man type tmpfs (rw,relatime) none on /var/log type tmpfs (rw,relatime) none on /var/lib/dbus type tmpfs (rw,relatime) none on /tmp type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) none on /var/tmp type tmpfs (rw,relatime) none on /var/lib/NetworkManager type tmpfs (rw,relatime) none on /var/lib/systemd/random-seed type tmpfs (rw,relatime) none on /var/spool type tmpfs (rw,relatime) none on /var/lib/nfs type tmpfs (rw,relatime) none on /var/lib/gssproxy type tmpfs (rw,relatime) none on /var/lib/logrotate type tmpfs (rw,relatime) none on /etc type tmpfs (rw,relatime) none on /var/lib/rsyslog type tmpfs (rw,relatime) none on /root/.ssh type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) 192.168.200.253:/exports/state/oleg110-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.10,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg110-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.10,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg110-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.10,local_lock=none,addr=192.168.200.253) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=378180k,mode=700) 192.168.200.253:/exports/testreports/64166/testresults/sanityn-zfs-rocky8.10_x86_64-rocky8.10_x86_64 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.201.10,local_lock=none,addr=192.168.200.253) /dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime) Checking servers environments Checking clients oleg110-client.virtnet environments /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg110-server' oleg110-server: oleg110-server.virtnet: executing load_modules_local oleg110-server: Loading modules from /home/green/git/lustre-release/lustre oleg110-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy oleg110-server: detected 4 online CPUs by sysfs oleg110-server: MODOPTS_LIBCFS= oleg110-server: Force libcfs to create 2 CPU partitions Setup mgs, mdt, osts Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg110-server: oleg110-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg110-client: oleg110-server: ssh exited with exit code 1 Started lustre-MDT0000 Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg110-server: oleg110-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg110-client: oleg110-server: ssh exited with exit code 1 Started lustre-OST0000 Start ost2: mount -t lustre -o localrecov lustre-ost2/ost2 /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg110-server: oleg110-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg110-client: oleg110-server: ssh exited with exit code 1 Started lustre-OST0001 Starting client: oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre Starting client oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre Started clients oleg110-client.virtnet: 192.168.201.110@tcp:/lustre on /mnt/lustre type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) Starting client: oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre2 Starting client oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre2 Started clients oleg110-client.virtnet: 192.168.201.110@tcp:/lustre on /mnt/lustre2 type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff9ae584d36800.idle_timeout=debug osc.lustre-OST0000-osc-ffff9ae588e85800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9ae584d36800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9ae588e85800.idle_timeout=debug disable quota as required ost.OSS.ost_io.nrs_policies=tbf projid ost.OSS.ost_io.nrs_tbf_rule=start ext_proj_rw projid={100} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.1498 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.275157 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.323496 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.306981 s, 3.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.426769 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.3636 s, 2.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.365066 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.400015 s, 2.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.316762 s, 3.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.492417 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.40195 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.358034 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.419209 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.351397 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.381203 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.46747 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.604776 s, 1.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.424282 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.379193 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.40312 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.415019 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.404239 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.414169 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.455394 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.431906 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.499218 s, 2.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.439564 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.451893 s, 2.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.431233 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.455481 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.486919 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.568332 s, 1.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.574209 s, 1.8 MB/s removed '/mnt/lustre/d77kh.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77kh.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77kh.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 14.8764 s, 7.0 MB/s Write runtime is 16 s, speed is 6.250000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.60908 s, 18.7 MB/s Read runtime is 7 s, speed is 14.285714 IOPS cleanup_tbf /mnt/lustre/d77kh.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_proj_rw ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=tbf projid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w projid={100}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r projid={100}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.20876 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0940396 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171119 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16994 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163714 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.246743 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136456 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.326724 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240508 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114538 s, 9.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.278234 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.338174 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.368306 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.257043 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.374062 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.414293 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0473094 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0836315 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119379 s, 87.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0890006 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0774626 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155231 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107366 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191184 s, 5.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166134 s, 6.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110701 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121579 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121514 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129051 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121049 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115324 s, 9.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239189 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137751 s, 7.6 MB/s removed '/mnt/lustre/d77kh.sanityn/nrs_r_oleg110-client.virtnet' removed '/mnt/lustre/d77kh.sanityn/nrs_w_oleg110-client.virtnet' removed directory '/mnt/lustre/d77kh.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 13.3509 s, 7.9 MB/s Write runtime is 15 s, speed is 6.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.1829 s, 10.3 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kh.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_policies=fifo PASS 77kh (294s) == sanityn test 77ki: Add rule with unsupported TBF type should fail for generic TBF ========================================================== 07:35:35 (1779708935) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_tbf_rule=start ext_fail projid={100}&opcode={ost_write} rate=20: Invalid argument oleg110-server: error: set_param: setting 'ost/OSS/ost_io/nrs_tbf_rule'='start ext_fail projid={100}&opcode={ost_write} rate=20': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_policies=fifo PASS 77ki (25s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 07:36:00 (1779708960) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (15s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 07:36:15 (1779708975) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg110-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 4.05208 s, 259 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg110-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.2137 s, 261 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (64s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 07:37:19 (1779709039) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.718234 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104855 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0909439 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140657 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147205 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.466056 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.42824 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.520358 s, 2.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.61647 s, 1.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.554896 s, 1.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.557776 s, 1.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.575561 s, 1.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.627499 s, 1.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.664098 s, 1.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.641079 s, 1.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191446 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298527 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118168 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111647 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902009 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109069 s, 96.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168849 s, 6.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14044 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251254 s, 4.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.191087 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193489 s, 5.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199463 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23036 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271362 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.816945 s, 1.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.53939 s, 1.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.835969 s, 1.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0760052 s, 13.8 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 15.7746 s, 6.6 MB/s Write runtime is 16 s, speed is 6.250000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.12003 s, 17.1 MB/s Read runtime is 7 s, speed is 14.285714 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.14576 s, 14.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180959 s, 5.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.231255 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.243549 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.267641 s, 3.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.312944 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.314455 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.325844 s, 3.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.422843 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.354945 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.403272 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.431892 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.319246 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.428505 s, 2.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9ae584d36800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff9ae584d36800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.448563 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.356319 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.514311 s, 2.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232823 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.258161 s, 4.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19667 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199665 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216031 s, 4.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.225996 s, 4.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224625 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240931 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264759 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.267484 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.326433 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.333751 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.364728 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.337438 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.274819 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.284236 s, 3.7 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg110-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg110-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 11.6884 s, 9.0 MB/s Write runtime is 13 s, speed is 7.692307 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.01924 s, 17.4 MB/s Read runtime is 8 s, speed is 12.500000 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (123s) == sanityn test 77o: Changing rank should not panic ====== 07:39:23 (1779709163) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (21s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 07:39:43 (1779709183) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (175s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 07:42:39 (1779709359) mds.MDS.mdt.nrs_policies=tbf oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg110-client: oleg110-server: ssh exited with exit code 36 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 oleg110-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg110-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg110-client: oleg110-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (60s) == sanityn test 77r: Change type of tbf policy at run time ========================================================== 07:43:38 (1779709418) mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 269188 kill md_thread xargs PASS 77r (77s) == sanityn test 77s: Check TBF LRU shrinker ============== 07:44:55 (1779709495) jobid_name=%e.%p jobid_var=TEST_77s_JOBVAR mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start touch jobid={touch.*} rate=1000 mds.MDS.mdt.nrs_tbf_rule=start unlink jobid={unlink.*} rate=1000 touch jobid={touch.*} 1000, ref 10 touch jobid={touch.*} 1000, ref 0 unlink jobid={unlink.*} 1000, ref 10 unlink jobid={unlink.*} 1000, ref 0 mds.MDS.mdt.nrs_policies=fifo jobid_var=procname_uid jobid_name=%e.%u PASS 77s (79s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 07:46:15 (1779709575) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_policies=fifo PASS 78 (16s) == sanityn test 79: xattr: intent error ================== 07:46:30 (1779709590) File: /mnt/lustre/d79.sanityn Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115238826935707 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-25 07:46:33.000000000 -0400 Modify: 2026-05-25 07:46:33.000000000 -0400 Change: 2026-05-25 07:46:33.000000000 -0400 Birth: 2026-05-25 07:46:34.000000000 -0400 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (23s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 07:46:53 (1779709613) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (5s) == sanityn test 80b: Accessing directory during migration ========================================================== 07:46:58 (1779709618) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (5s) == sanityn test 81a: rename and stat under striped directory ========================================================== 07:47:04 (1779709624) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (4s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 07:47:09 (1779709629) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (5s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 07:47:14 (1779709634) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (5s) == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 07:47:20 (1779709640) - open/close 525 (time 1779709659.19 total 10.04 last 52.31) - open/close 1112 (time 1779709669.20 total 20.05 last 58.64) - open/close 1704 (time 1779709679.22 total 30.07 last 59.06) - open/close 2174 (time 1779709689.22 total 40.07 last 46.99) total: 2500 open/close in 46.77 seconds: 53.45 ops/second snapshot_time 1779709828.489289680 secs.nsecs start_time 1779709646.076423686 secs.nsecs elapsed_time 182.412865994 secs.nsecs open 2505 samples [usecs] 216 26729 4935157 17752642349 close 2504 samples [usecs] 173 10678 1107685 1173954303 mknod 2501 samples [usecs] 644 26613 4365535 14845181799 unlink 2500 samples [usecs] 739 172645 7557428 125322384638 mkdir 2 samples [usecs] 1055 1510 2565 3393125 rmdir 2 samples [usecs] 10374 17606 27980 417591112 rename 2500 samples [usecs] 411 91008 25207568 539366017006 getattr 3763 samples [usecs] 7 10689 342833 621112273 parallel_rename_file 2500 samples [usecs] 411 91008 25207568 539366017006 crossdir_rename 2500 samples [usecs] 411 91008 25207568 539366017006 PASS 81d (195s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 07:50:35 (1779709835) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7601 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7601 PASS 82 (10s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 07:50:45 (1779709845) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (5s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 07:50:50 (1779709850) fail_loc=0x8000060b PASS 84 (16s) == sanityn test 85: Lustre API root cache race =========== 07:51:06 (1779709866) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1779709868 Finishing test test1 at 1779709874 PASS 85 (16s) == sanityn test 90: open/create and unlink striped directory ========================================================== 07:51:22 (1779709882) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (5s) == sanityn test 91: chmod and unlink striped directory === 07:51:28 (1779709888) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (6s) == sanityn test 92: create remote directory under orphan directory ========================================================== 07:51:33 (1779709893) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (5s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 07:51:38 (1779709898) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5818 0x16ba 0x240000400 1 5784 0x1698 0x280000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5819 0x16bb 0x240000400 1 5785 0x1699 0x280000400 rc1=0 and rc2=0 PASS 93 (24s) == sanityn test 94: signal vs CP callback race =========== 07:52:02 (1779709922) osc.lustre-OST0000-osc-ffff9ae584d36800.idle_timeout=0 osc.lustre-OST0000-osc-ffff9ae588e85800.idle_timeout=0 osc.lustre-OST0001-osc-ffff9ae584d36800.idle_timeout=0 osc.lustre-OST0001-osc-ffff9ae588e85800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.231956 s, 35.3 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 2.99677 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 6178: 286628 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0629323 s, 65.1 kB/s osc.lustre-OST0000-osc-ffff9ae584d36800.idle_timeout=debug osc.lustre-OST0000-osc-ffff9ae588e85800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9ae584d36800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9ae588e85800.idle_timeout=debug PASS 94 (20s) == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 07:52:22 (1779709942) llite.lustre-ffff9ae584d36800.fast_read=0 llite.lustre-ffff9ae588e85800.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0619495 s, 33.9 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff9ae584d36800.fast_read=1 llite.lustre-ffff9ae588e85800.fast_read=1 PASS 95a (24s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 07:52:46 (1779709966) llite.lustre-ffff9ae584d36800.fast_read=0 llite.lustre-ffff9ae588e85800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.00745048 s, 1.6 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff9ae584d36800.fast_read=1 llite.lustre-ffff9ae588e85800.fast_read=1 PASS 95b (18s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 07:53:04 (1779709984) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (6s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 07:53:10 (1779709990) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0140124 s, 292 kB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (12s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 07:53:23 (1779710003) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.231647 s, 9.1 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (11s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 07:53:34 (1779710014) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.198486 s, 10.6 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (11s) == sanityn test 100e: DoM: read on open and file size ==== 07:53:45 (1779710025) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.0197891 s, 606 kB/s PASS 100e (11s) == sanityn test 101a: Discard DoM data on unlink ========= 07:53:57 (1779710037) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0123421 s, 332 kB/s PASS 101a (14s) == sanityn test 101b: Discard DoM data on rename ========= 07:54:10 (1779710050) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0145538 s, 281 kB/s PASS 101b (13s) == sanityn test 101c: Discard DoM data on close-unlink === 07:54:23 (1779710063) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0108942 s, 376 kB/s PASS 101c (11s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 07:54:37 (1779710077) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1779710082 lockahead_test: lockahead_test.c:1171: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1779710086 Finishing test test23 at 1779710086 PASS 103 (20s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 07:54:57 (1779710097) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (6s) == sanityn test 105: Glimpse and lock cancel race ======== 07:55:03 (1779710103) fail_loc=0x416 PASS 105 (27s) == sanityn test 106a: Verify the btime via statx() ======= 07:55:31 (1779710131) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (5s) == sanityn test 106b: Glimpse RPCs test for statx ======== 07:55:36 (1779710136) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0217177 s, 47.2 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115238826938242 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-25 07:55:38.000000000 -0400 Modify: 2026-05-25 07:55:38.000000000 -0400 Change: 2026-05-25 07:55:38.000000000 -0400 Birth: 2026-05-25 07:55:39.000000000 -0400 /mnt/lustre/f106b.sanityn 144115238826938242 -rw-r--r-- 1 0 0 1779710139 1779710138 1779710138 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115238826938242 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-25 07:55:38.000000000 -0400 Modify: 2026-05-25 07:55:38.000000000 -0400 Change: 2026-05-25 07:55:38.000000000 -0400 Birth: 2026-05-25 07:55:39.000000000 -0400 1779710138 1024 11 PASS 106b (12s) == sanityn test 106c: Verify statx attributes mask ======= 07:55:49 (1779710149) PASS 106c (11s) == sanityn test 107a: Basic grouplock conflict =========== 07:56:00 (1779710160) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.678885 s, 15.4 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 5791 0x169f 0x280000400 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 First grouplock blocks second one PASS 107a (13s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 07:56:13 (1779710173) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.30003 s, 34.9 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5828 0x16c4 0x240000400 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7601 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (16s) == sanityn test 108a: lseek: parallel updates ============ 07:56:29 (1779710189) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.0155717 s, 642 kB/s llite.lustre-ffff9ae584d36800.hybrid_io=0 llite.lustre-ffff9ae588e85800.hybrid_io=0 fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 5.18138 s, 1.6 MB/s PASS 108a (15s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 07:56:45 (1779710205) 192.168.201.110@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg110-client.virtnet /mnt/lustre (opts:) 192.168.201.110@tcp:/lustre /mnt/lustre2 lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg110-client.virtnet /mnt/lustre2 (opts:) Starting race between client mounts (duration 30) Iteration 0 Starting client: oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre Starting client: oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre2 192.168.201.110@tcp:/lustre /mnt/lustre2 lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.110@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg110-client.virtnet /mnt/lustre (opts:) Stopping client oleg110-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' loading modules on: 'oleg110-server' oleg110-server: oleg110-server.virtnet: executing load_modules_local oleg110-server: Loading modules from /home/green/git/lustre-release/lustre oleg110-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1042: echo: write error: Device or resource busy oleg110-server: detected 4 online CPUs by sysfs oleg110-server: MODOPTS_LIBCFS= oleg110-server: Force libcfs to create 2 CPU partitions Starting client: oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre Starting client: oleg110-client.virtnet: -o user_xattr,flock 192.168.201.110@tcp:/lustre /mnt/lustre2 PASS 109 (61s) == sanityn test 110: do not grant another lock on resend ========================================================== 07:57:46 (1779710266) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115238860488707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-25 07:57:49.000000000 -0400 Modify: 2026-05-25 07:57:49.000000000 -0400 Change: 2026-05-25 07:57:49.000000000 -0400 Birth: 2026-05-25 07:57:50.000000000 -0400 fail_loc=0 fail_val=0 Connected clients: oleg110-client.virtnet oleg110-client.virtnet PASS 110 (75s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 07:59:01 (1779710341) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (4s) == sanityn test 112: update max-inherit in default LMV === 07:59:05 (1779710345) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (4s) == sanityn test 113: check servers of specified fs ======= 07:59:09 (1779710349) lustre-OST0000-osc-ffff9ae591683000 active. lustre-OST0001-osc-ffff9ae591683000 active. lustre-OST0000-osc-ffff9ae591683000 active. lustre-OST0001-osc-ffff9ae591683000 active. MGC192.168.201.110@tcp active. PASS 113 (10s) == sanityn test 114: implicit default LMV inherit ======== 07:59:19 (1779710359) SKIP: sanityn test_114 We need at least 2 MDTs for this test SKIP 114 (5s) == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 07:59:24 (1779710364) SKIP: sanityn test_115 ldiskfs only test SKIP 115 (5s) == sanityn test 116: DNE: Set default LMV layout from a remote client ========================================================== 07:59:29 (1779710369) SKIP: sanityn test_116 needs >= 2 MDTs SKIP 116 (5s) == sanityn test 121: trunc append race =================== 07:59:35 (1779710375) 1+0 records in 1+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.0179723 s, 1.1 MB/s fail_loc=0x80001436 1+0 records in 1+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.117152 s, 175 kB/s 0000000 0a79 0a79 0a79 0a79 0a79 0a79 0a79 0a79 * 0005000 0000 0000 0000 0000 0000 0000 0000 0000 * 000c800 0a79 0a79 0a79 0a79 0a79 0a79 0a79 0a79 * 0011800 PASS 121 (12s) == sanityn test 200: service remains healthy while able to process request ========================================================== 07:59:47 (1779710387) at_max=10 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=9 fail_loc=0x1e1 fail_val=20 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.21 s, 0.2 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.2545 s, 0.2 kB/s4096 bytes (4.1 kB, 4.0 KiB) copied, 20.3022 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.2766 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.2727 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.3138 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.3336 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.2465 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.2856 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.3074 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.3114 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.3427 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.3843 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.4699 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.4778 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.5295 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.572 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.5102 s, 0.1 kB/s at_max=600 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=116 oleg110-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9ae584d31800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff9ae584d31800.ost_server_uuid in IDLE state after 0 sec PASS 200 (90s) cleanup: ====================================================== == sanityn test complete, duration 12840 sec ============= 08:01:21 (1779710481) === sanityn: start cleanup 08:01:24 (1779710484) === Stopping clients: oleg110-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg110-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 08:07:06 (1779710826) ===