-----============= acceptance-small: sanityn ============----- Mon Aug 10 04:57:04 EDT 2026 MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg410-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 40a skipping tests SLOW=no: 33a === sanityn: start setup 04:57:21 (1786352241) === Starting client oleg410-client.virtnet: -o user_xattr,flock 192.168.204.110@tcp:/lustre /mnt/lustre2 Mount client oleg410-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.204.110@tcp:/lustre /mnt/lustre2 Started clients oleg410-client.virtnet: 192.168.204.110@tcp:/lustre on /mnt/lustre2 type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) oleg410-client.virtnet: executing check_config_client /mnt/lustre oleg410-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg410-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff98c8c2eba800.idle_timeout=debug osc.lustre-OST0000-osc-ffff98c8d1e4f800.idle_timeout=debug osc.lustre-OST0001-osc-ffff98c8c2eba800.idle_timeout=debug osc.lustre-OST0001-osc-ffff98c8d1e4f800.idle_timeout=debug disable quota as required oleg410-server: oleg410-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 04:57:51 (1786352271) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250232 s, 41.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7638] == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 04:57:53 (1786352273) PASS 0 (8s) == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 04:58:01 (1786352281) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (7s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 04:58:08 (1786352288) -rw-r--r-- 1 root root 0 Aug 10 04:58 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (8s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 04:58:16 (1786352296) -rw-r--r-- 1 root root 0 Aug 10 04:58 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (7s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 04:58:23 (1786352303) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (8s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 04:58:31 (1786352311) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (7s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 04:58:38 (1786352318) total 1 -rwxrwxrwx 1 root root 0 Aug 10 04:58 f2a -rwxrwxrwx 1 root root 0 Aug 10 04:58 f2b total 1 -rwxrwxrwx 1 root root 0 Aug 10 04:58 f2a -rwxrwxrwx 1 root root 0 Aug 10 04:58 f2b running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.0193364 s, 26.5 kB/s PASS 2e (7s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 04:58:45 (1786352325) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (3s) == sanityn test 2g: check blocks update on sync write ==== 04:58:49 (1786352329) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.471493 s, 4.4 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (8s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 04:58:57 (1786352337) PASS 3 (8s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 04:59:05 (1786352345) PASS 4 (8s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 04:59:13 (1786352353) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (7s) == sanityn test 6: remove of open file on other node ============================================================================ 04:59:20 (1786352360) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (8s) == sanityn test 7: remove of open directory on other node ======================================================================= 04:59:28 (1786352368) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (8s) == sanityn test 8: remove of open special file on other node ==================================================================== 04:59:36 (1786352376) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (9s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 04:59:45 (1786352385) PASS 9a (8s) == sanityn test 9b: append to striped sparse file ======== 04:59:53 (1786352393) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.059236 s, 17.7 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.0646843 s, 0.0 kB/s Data read (expecting 'foo'): foo PASS 9b (8s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 05:00:01 (1786352401) 1+0 records in 1+0 records out 1 byte copied, 0.0115795 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0152818 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0181432 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.031021 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0142211 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0264532 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0187896 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0141015 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0145419 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0119103 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0140035 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0121609 s, 0.1 kB/s PASS 10a (9s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 05:00:10 (1786352410) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.0254893 s, 121 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0552953 s, 74.1 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.000304375 s, 10.1 MB/s PASS 10b (8s) == sanityn test 11: execution of file opened for write should return error ============================================================== 05:00:18 (1786352418) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 298: /mnt/lustre2/d11/f: Text file busy PASS 11 (7s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 05:00:26 (1786352426) warning: '-runtime' deprecated, use '-t runtime' instead - link 309 (time 1786352438.28 total 10.01 last 30.88) - link 574 (time 1786352448.28 total 20.01 last 26.49) - link 810 (time 1786352458.29 total 30.02 last 23.58) - link 1069 (time 1786352468.32 total 40.05 last 25.83) - link 1348 (time 1786352478.34 total 50.07 last 27.84) - link 1611 (time 1786352488.35 total 60.08 last 26.29) - link 1909 (time 1786352498.37 total 70.09 last 29.75) - link 2191 (time 1786352508.39 total 80.12 last 28.12) - link 2475 (time 1786352518.41 total 90.14 last 28.35) total: 2733 link in 100.04 seconds: 27.32 ops/second using seed 2832021079 running for 100 seconds - stat 3758 (time 1786352440 ; total 11 ; last 11) - stat 8145 (time 1786352451 ; total 22 ; last 11) - stat 10000 (time 1786352456 ; total 27 ; last 5) - stat 13475 (time 1786352467 ; total 38 ; last 11) - stat 17890 (time 1786352478 ; total 49 ; last 11) - stat 19680 (time 1786352489 ; total 60 ; last 11) - stat 20000 (time 1786352491 ; total 62 ; last 2) - stat 21913 (time 1786352502 ; total 73 ; last 11) - stat 23825 (time 1786352513 ; total 84 ; last 11) - stat 25636 (time 1786352524 ; total 95 ; last 11) total: 26473 stats in 100 seconds: 264.730011 stats/second - unlinked 0 (time 1786352479 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile2733) error: No such file or directory total: 2732 unlinks in 83 seconds: 32.915661 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (21436) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (21438) - No such process PASS 12 (142s) == sanityn test 13: test directory page revocation ======= 05:02:47 (1786352567) f13 f13 PASS 13 (8s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 05:02:55 (1786352575) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 333: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 5089: 22636 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (7s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 05:03:02 (1786352582) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 5089: 23215 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (7s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 05:03:10 (1786352590) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 348: 23793 Terminated $DIR1/$tdir/sleep 60 PASS 14b (8s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 05:03:17 (1786352597) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 360: 24375 Terminated $DIR1/$tdir/sleep 60 PASS 14c (8s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 05:03:25 (1786352605) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 371: 24958 Terminated $DIR1/$tdir/sleep 60 PASS 14d (10s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 05:03:35 (1786352615) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky sleep 5 for ZFS MDS Waiting for MDT destroys to complete fallocate on zfs doesn't consume space STRIPECOUNT=2 ORIGFREE=7531520 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7531520KB free > 800000KB, increase MAXFREE (or reduce fs size) sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (50s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 05:04:26 (1786352666) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.55694 s, 15.1 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.414236 s, 20.3 MB/s Chance of close/open is 1 in 50 Seed set to 2669 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x31dc1 000001[1] 1786352670.614549 trunc from 00000000 to 0x031dc0 (0x31dc1 bytes) truncating to largest ever: 0x97d534 000003[1] 1786352670.663399 trunc from 0x031dc1 to 0x97d533 (0x94b773 bytes) 000004[1] 1786352670.697310 trunc from 0x97d534 to 0x793feb (0x1e9548 bytes) 000006[0] 1786352670.788031 mapread 0x5d642d thru 0x5e0013 (0x9be7 bytes) 000007[0] 1786352670.852804 trunc from 0x793fec to 0x1805ab (0x613a40 bytes) 000011[1] 1786352670.979798 trunc from 0x1805ac to 0x1e83f3 (0x67e48 bytes) 000013[0] 1786352671.087263 trunc from 0x1e83f4 to 0x5de597 (0x3f61a4 bytes) 000014[1] 1786352671.173780 read 0x175a75 thru 0x17eefb (0x9487 bytes) 000015[1] 1786352671.213196 mapread 0x2e9b82 thru 0x2ea7dd (0xc5c bytes) 000016[0] 1786352671.249599 write 0x0bae2f thru 0x0c31e3 (0x83b5 bytes) 000018[1] 1786352671.315803 read 0x1ff424 thru 0x20e844 (0xf421 bytes) 000019[1] 1786352671.381563 trunc from 0x5de598 to 0x6def53 (0x1009bc bytes) 000020[1] 1786352671.472289 read 0x56dc74 thru 0x579e03 (0xc190 bytes) 000021[1] 1786352671.523645 read 0x0cdc73 thru 0x0d0eda (0x3268 bytes) 000023[0] 1786352671.584462 read 0x018f5f thru 0x027f9f (0xf041 bytes) 000024[1] 1786352671.643279 read 0x55118c thru 0x55bd5f (0xabd4 bytes) 000027[1] 1786352671.719104 mapwrite 0x6d073b thru 0x6d3c29 (0x34ef bytes) 000028[0] 1786352671.848618 write 0x4d8630 thru 0x4db874 (0x3245 bytes) 000030[1] 1786352671.907364 mapread 0x023041 thru 0x02f63c (0xc5fc bytes) 000032[1] 1786352671.978342 mapread 0x34cd8e thru 0x35a54b (0xd7be bytes) 000033[0] 1786352672.066605 write 0x4e69f0 thru 0x4f492d (0xdf3e bytes) 000034[1] 1786352672.103977 trunc from 0x6def54 to 0x5123ff (0x1ccb54 bytes) 000035[1] 1786352672.203044 read 0x25ac25 thru 0x262521 (0x78fd bytes) 000036[0] 1786352672.242442 mapread 0x019e30 thru 0x027c6b (0xde3c bytes) 000038[0] 1786352672.352626 read 0x29edb6 thru 0x29f8d0 (0xb1b bytes) 000042[0] 1786352672.429155 read 0x42408c thru 0x432843 (0xe7b8 bytes) 000044[1] 1786352672.460216 mapread 0x4da5a1 thru 0x4e5b28 (0xb588 bytes) 000047[1] 1786352672.492136 mapwrite 0x8af937 thru 0x8b9cb4 (0xa37e bytes) truncating to largest ever: 0x9b17c3 000048[0] 1786352672.703132 trunc from 0x8b9cb5 to 0x9b17c2 (0xf7b0e bytes) 000051[1] 1786352672.851967 read 0x73f94c thru 0x7450f7 (0x57ac bytes) 000054[0] 1786352672.928940 mapwrite 0x5e11ec thru 0x5e3fb2 (0x2dc7 bytes) 000055[0] 1786352673.032406 read 0x7ea8ce thru 0x7f6de4 (0xc517 bytes) 000057[1] 1786352673.138454 write 0x781c6a thru 0x787e19 (0x61b0 bytes) 000061[1] 1786352673.262010 read 0x27140a thru 0x273447 (0x203e bytes) 000062[1] 1786352673.296342 read 0x9123fd thru 0x918497 (0x609b bytes) 000065[0] 1786352673.322007 mapwrite 0x303d15 thru 0x310fdb (0xd2c7 bytes) 000067[0] 1786352673.510420 write 0x6916dc thru 0x69942b (0x7d50 bytes) 000068[0] 1786352673.595991 mapread 0x62db3b thru 0x63cac8 (0xef8e bytes) 000069[0] 1786352673.671978 mapwrite 0x64e693 thru 0x655fd8 (0x7946 bytes) 000073[1] 1786352673.858049 write 0x912d3a thru 0x918d1d (0x5fe4 bytes) 000074[1] 1786352673.903057 trunc from 0x9b17c3 to 0x4e8b2b (0x4c8c97 bytes) 000075[0] 1786352673.953579 mapwrite 0x4ca0c2 thru 0x4cf13e (0x507d bytes) 000076[0] 1786352674.096395 write 0x0889f6 thru 0x092cc4 (0xa2cf bytes) 000077[1] 1786352674.131566 mapread 0x3fcd39 thru 0x40789f (0xab67 bytes) 000078[1] 1786352674.224867 write 0x286a3d thru 0x28c956 (0x5f1a bytes) 000079[1] 1786352674.253995 mapwrite 0x132dbc thru 0x13caa5 (0x9cea bytes) 000080[0] 1786352674.398231 mapwrite 0x2d2a36 thru 0x2d822e (0x57f9 bytes) 000081[1] 1786352674.568877 read 0x0ea4f2 thru 0x0eb347 (0xe56 bytes) 000086[0] 1786352674.599766 read 0x371729 thru 0x373487 (0x1d5f bytes) truncating to largest ever: 0x9e4e83 000093[1] 1786352674.611982 trunc from 0x4e8b2c to 0x9e4e82 (0x4fc357 bytes) 000096[1] 1786352674.717462 mapwrite 0x1a98bc thru 0x1ae8ef (0x5034 bytes) 000099[0] 1786352674.853299 trunc from 0x9e4e83 to 0x2bd903 (0x72757f bytes) 000100[0] 1786352674.906562 trunc from 0x2bd904 to 0x8a3ab5 (0x5e61b2 bytes) 000101[0] 1786352674.938884 trunc from 0x8a3ab6 to 0x1f7df4 (0x6abcc1 bytes) 000103[0] 1786352674.994752 write 0x2a57b3 thru 0x2b39bb (0xe209 bytes) 000105[0] 1786352675.026504 mapwrite 0x2dc6b4 thru 0x2e4aa9 (0x83f6 bytes) 000106[0] 1786352675.152519 write 0x8ef98c thru 0x8fd620 (0xdc95 bytes) 000107[1] 1786352675.164330 mapread 0x46f64b thru 0x47012b (0xae1 bytes) 000109[0] 1786352675.247854 trunc from 0x8fd621 to 0x169891 (0x793d8f bytes) 000110[0] 1786352675.331964 read 0x01e22c thru 0x0230c3 (0x4e98 bytes) 000112[1] 1786352675.396914 trunc from 0x169892 to 0x814fcc (0x6ab73b bytes) 000115[0] 1786352675.485924 mapread 0x0ca5a3 thru 0x0cb1b3 (0xc11 bytes) 000116[1] 1786352675.627989 read 0x668238 thru 0x66b5ce (0x3397 bytes) 000119[0] 1786352675.681075 read 0x702728 thru 0x70c833 (0xa10c bytes) 000125[0] 1786352675.711094 mapread 0x54ddcc thru 0x55ab7c (0xcdb1 bytes) 000126[1] 1786352675.750609 mapwrite 0x46194d thru 0x464030 (0x26e4 bytes) 000128[1] 1786352675.886123 mapread 0x2a2916 thru 0x2a4893 (0x1f7e bytes) 000129[0] 1786352675.934944 mapwrite 0x7ad7d5 thru 0x7babe5 (0xd411 bytes) 000130[1] 1786352676.057073 trunc from 0x814fcd to 0x8ed689 (0xd86bd bytes) 000131[0] 1786352676.127428 read 0x48c219 thru 0x491858 (0x5640 bytes) 000134[1] 1786352676.197610 trunc from 0x8ed68a to 0x024fcb (0x8c86be bytes) 000138[0] 1786352676.328935 read 0x02411e thru 0x024fcb (0xeae bytes) 000139[1] 1786352676.356369 read 0x01bd92 thru 0x01c0f9 (0x368 bytes) 000142[0] 1786352676.393477 trunc from 0x024fcc to 0x1a46a6 (0x17f6db bytes) 000144[1] 1786352676.484024 read 0x0ebbcd thru 0x0fafa5 (0xf3d9 bytes) 000145[0] 1786352676.595244 mapread 0x02d6ab thru 0x036787 (0x90dd bytes) 000149[1] 1786352676.736858 write 0x53f213 thru 0x53f560 (0x34e bytes) 000150[0] 1786352676.752128 mapwrite 0x1eddbf thru 0x1fd517 (0xf759 bytes) 000151[1] 1786352677.354972 write 0x8b0ecf thru 0x8bb97b (0xaaad bytes) 000152[1] 1786352677.373926 write 0x7d2b88 thru 0x7d467b (0x1af4 bytes) 000154[0] 1786352677.485558 mapread 0x611071 thru 0x61d8a4 (0xc834 bytes) 000156[1] 1786352677.660565 write 0x2a0728 thru 0x2a258c (0x1e65 bytes) 000158[0] 1786352677.699465 mapread 0x5eb76c thru 0x5f9956 (0xe1eb bytes) 000159[1] 1786352677.821115 mapwrite 0x2a6671 thru 0x2ab850 (0x51e0 bytes) 000169[1] 1786352677.985521 mapwrite 0x753fef thru 0x7553c4 (0x13d6 bytes) 000170[1] 1786352678.197929 trunc from 0x8bb97c to 0x2a49d2 (0x616fa9 bytes) 000175[0] 1786352678.313398 mapread 0x23f02c thru 0x248037 (0x900c bytes) 000176[1] 1786352678.458656 mapwrite 0x5e4f4e thru 0x5e79a3 (0x2a56 bytes) 000177[0] 1786352678.643038 read 0x5c7ab4 thru 0x5d25ff (0xab4c bytes) 000178[1] 1786352678.691610 read 0x470491 thru 0x47e3b3 (0xdf23 bytes) 000181[0] 1786352678.748060 write 0x3f5591 thru 0x3f9137 (0x3ba7 bytes) 000182[0] 1786352678.776513 write 0x63a948 thru 0x648483 (0xdb3c bytes) 000183[0] 1786352678.818625 read 0x340793 thru 0x345ea1 (0x570f bytes) 000186[0] 1786352678.867452 mapread 0x4e1d54 thru 0x4e324f (0x14fc bytes) 000187[1] 1786352678.889376 write 0x2504be thru 0x25eed8 (0xea1b bytes) 000188[0] 1786352678.961399 read 0x5a6d58 thru 0x5ac069 (0x5312 bytes) 000190[0] 1786352679.062296 trunc from 0x648484 to 0x186859 (0x4c1c2a bytes) 000195[1] 1786352679.281518 read 0x02cf6e thru 0x03096c (0x39ff bytes) 000196[0] 1786352679.387142 trunc from 0x18685a to 0x0322cd (0x15458c bytes) 000197[0] 1786352679.532234 write 0x4834b0 thru 0x48e623 (0xb174 bytes) 000198[0] 1786352679.559543 mapwrite 0x260f57 thru 0x26ac30 (0x9cda bytes) 000199[0] 1786352679.685040 mapread 0x0b26e4 thru 0x0b7c4a (0x5567 bytes) 000201[0] 1786352679.738148 mapwrite 0x51f0d0 thru 0x52cca6 (0xdbd7 bytes) 000202[0] 1786352679.907366 trunc from 0x52cca7 to 0x041cb6 (0x4eaff0 bytes) 000203[0] 1786352679.927171 mapwrite 0x9138ec thru 0x916469 (0x2b7e bytes) 000205[1] 1786352680.028960 trunc from 0x91646a to 0x5b2ea6 (0x3635c3 bytes) 000207[0] 1786352680.098593 write 0x34d343 thru 0x3523d8 (0x5096 bytes) 000210[1] 1786352680.192349 trunc from 0x5b2ea7 to 0x0ae618 (0x50488e bytes) 000211[0] 1786352680.274353 mapread 0x041f9f thru 0x04b421 (0x9483 bytes) 000213[1] 1786352680.340210 mapread 0x012985 thru 0x0176a7 (0x4d23 bytes) 000214[1] 1786352680.374898 read 0x0448f3 thru 0x0475ef (0x2cfd bytes) 000217[0] 1786352680.420279 mapwrite 0x889a8f thru 0x88bf6b (0x24dd bytes) 000218[1] 1786352680.621445 trunc from 0x88bf6c to 0x27104e (0x61af1d bytes) 000219[0] 1786352680.678378 mapwrite 0x0dced1 thru 0x0e285f (0x598f bytes) 000220[0] 1786352680.817372 mapread 0x1e58a6 thru 0x1ec1e3 (0x693e bytes) 000221[0] 1786352680.871576 read 0x065069 thru 0x07027e (0xb216 bytes) 000223[0] 1786352680.963550 mapwrite 0x5c773f thru 0x5cb05c (0x391e bytes) 000224[0] 1786352681.158991 mapread 0x46b2e8 thru 0x473d9e (0x8ab7 bytes) 000225[0] 1786352681.165471 mapwrite 0x3f3147 thru 0x3f75ab (0x4465 bytes) 000226[1] 1786352681.265470 trunc from 0x5cb05d to 0x173909 (0x457753 bytes) 000229[1] 1786352681.402053 read 0x0ca632 thru 0x0d4ef7 (0xa8c6 bytes) 000232[0] 1786352681.548495 mapwrite 0x24fa63 thru 0x2546f4 (0x4c92 bytes) 000235[0] 1786352681.781468 mapread 0x11cd0a thru 0x11e3b3 (0x16aa bytes) 000237[0] 1786352681.870119 mapread 0x0bdf03 thru 0x0c4e1b (0x6f19 bytes) 000238[0] 1786352681.920000 read 0x02bd9f thru 0x034858 (0x8aba bytes) 000240[1] 1786352682.051412 write 0x5742a1 thru 0x57cb5f (0x88bf bytes) 000241[0] 1786352682.091408 write 0x0581c1 thru 0x05b98f (0x37cf bytes) 000242[1] 1786352682.111738 trunc from 0x57cb60 to 0x7f602c (0x2794cd bytes) 000244[0] 1786352682.207255 read 0x1a15a1 thru 0x1aac64 (0x96c4 bytes) 000245[0] 1786352682.240730 mapwrite 0x4fea4b thru 0x50519a (0x6750 bytes) 000246[1] 1786352682.361300 write 0x2acfc4 thru 0x2bb47b (0xe4b8 bytes) 000247[0] 1786352682.473797 mapwrite 0x92e4cc thru 0x939392 (0xaec7 bytes) 000248[1] 1786352682.643584 mapread 0x049ad3 thru 0x0544dc (0xaa0a bytes) 000250[1] 1786352682.790865 mapwrite 0x20e8f0 thru 0x212697 (0x3da8 bytes) 000254[0] 1786352682.985048 write 0x8013ee thru 0x8104e2 (0xf0f5 bytes) 000256[0] 1786352683.087440 trunc from 0x939393 to 0x4b2222 (0x487170 bytes) 000257[0] 1786352683.147337 read 0x178a2b thru 0x1882f2 (0xf8c8 bytes) 000258[1] 1786352683.190543 trunc from 0x4b2223 to 0x64e98b (0x19c769 bytes) 000259[0] 1786352683.241649 read 0x34f3bb thru 0x35592d (0x6573 bytes) 000264[0] 1786352683.324091 mapwrite 0x7ae01a thru 0x7b2a88 (0x4a6f bytes) 000265[1] 1786352683.495632 read 0x404f7c thru 0x4098b5 (0x493a bytes) 000267[1] 1786352683.579063 write 0x30037e thru 0x30da63 (0xd6e6 bytes) 000269[1] 1786352683.656405 mapread 0x43c849 thru 0x442e4f (0x6607 bytes) 000271[0] 1786352683.725554 write 0x9cbbd3 thru 0x9d0a06 (0x4e34 bytes) 000272[1] 1786352683.758304 trunc from 0x9d0a07 to 0x014fce (0x9bba38 bytes) 000274[1] 1786352683.835981 write 0x21a091 thru 0x21aaf9 (0xa69 bytes) 000275[0] 1786352683.875931 read 0x03f044 thru 0x047616 (0x85d3 bytes) 000276[0] 1786352683.990357 write 0x1ec4eb thru 0x1f4183 (0x7c99 bytes) 000280[1] 1786352684.146146 write 0x95ea17 thru 0x96579a (0x6d84 bytes) 000281[0] 1786352684.217296 write 0x7d49d1 thru 0x7ddb8b (0x91bb bytes) 000284[0] 1786352684.294549 trunc from 0x96579b to 0x856df6 (0x10e9a4 bytes) 000285[0] 1786352684.386375 mapwrite 0x22e0f2 thru 0x2384a6 (0xa3b5 bytes) 000286[0] 1786352684.577561 read 0x27f924 thru 0x2868b8 (0x6f95 bytes) 000289[0] 1786352684.658386 write 0x20e90a thru 0x211de3 (0x34da bytes) 000290[0] 1786352684.679235 write 0x8675e1 thru 0x86dea7 (0x68c7 bytes) 000292[1] 1786352684.707232 mapread 0x0fa41a thru 0x1089fb (0xe5e2 bytes) 000295[0] 1786352684.786779 mapwrite 0x16e8cf thru 0x1709b8 (0x20ea bytes) 000297[0] 1786352684.889367 mapwrite 0x4ea981 thru 0x4eb209 (0x889 bytes) 000299[1] 1786352684.977196 read 0x39ee64 thru 0x3a9fca (0xb167 bytes) 000300[1] 1786352684.998334 trunc from 0x86dea8 to 0x6c5314 (0x1a8b93 bytes) 000302[1] 1786352685.079420 write 0x35388a thru 0x3567b8 (0x2f2f bytes) 000303[1] 1786352685.091400 write 0x736ca8 thru 0x744093 (0xd3ec bytes) 000304[1] 1786352685.113530 mapread 0x323772 thru 0x332290 (0xeb1f bytes) 000305[1] 1786352685.147973 mapread 0x4d6303 thru 0x4d6928 (0x626 bytes) 000307[1] 1786352685.185650 mapread 0x1ec75b thru 0x1f8c40 (0xc4e6 bytes) 000308[1] 1786352685.248933 mapwrite 0x3351f6 thru 0x344991 (0xf79c bytes) 000309[1] 1786352685.323387 mapwrite 0x1fbb21 thru 0x1fc672 (0xb52 bytes) 000310[1] 1786352685.408820 mapwrite 0x08214e thru 0x0906fd (0xe5b0 bytes) 000317[0] 1786352685.745277 write 0x155a6f thru 0x158f73 (0x3505 bytes) 000318[0] 1786352685.789938 mapwrite 0x68d41b thru 0x69b7ab (0xe391 bytes) 000320[1] 1786352686.010712 trunc from 0x744094 to 0x954d2e (0x210c9b bytes) 000321[1] 1786352686.072085 mapread 0x43e3b9 thru 0x44797f (0x95c7 bytes) 000322[0] 1786352686.131956 read 0x20c348 thru 0x211e34 (0x5aed bytes) 000326[0] 1786352686.194005 trunc from 0x954d2f to 0x9ab02e (0x56300 bytes) 000331[0] 1786352686.322499 trunc from 0x9ab02f to 0x64b676 (0x35f9b8 bytes) 000332[0] 1786352686.375895 read 0x0cdd78 thru 0x0d9065 (0xb2ee bytes) 000333[1] 1786352686.399845 read 0x38bac0 thru 0x399840 (0xdd81 bytes) 000334[0] 1786352686.446550 mapread 0x1ca1c2 thru 0x1d8f5f (0xed9e bytes) 000337[1] 1786352686.586264 mapwrite 0x74cea8 thru 0x750084 (0x31dd bytes) 000338[0] 1786352686.759576 mapwrite 0x86ebc9 thru 0x875e5f (0x7297 bytes) 000340[1] 1786352686.972945 trunc from 0x875e60 to 0x1f8039 (0x67de26 bytes) 000342[0] 1786352687.017787 trunc from 0x1f803a to 0x6b91ce (0x4c1195 bytes) 000344[1] 1786352687.113290 mapwrite 0x625645 thru 0x6287a0 (0x315c bytes) 000349[1] 1786352687.302913 trunc from 0x6b91cf to 0x7e2a63 (0x129895 bytes) 000352[0] 1786352687.349010 write 0x181697 thru 0x187255 (0x5bbf bytes) 000353[1] 1786352687.402185 mapwrite 0x5636cd thru 0x56458d (0xec1 bytes) 000354[0] 1786352687.475338 trunc from 0x7e2a64 to 0x5df362 (0x203701 bytes) 000355[0] 1786352687.515858 trunc from 0x5df363 to 0x1d5b5a (0x409808 bytes) 000358[1] 1786352687.560737 read 0x0af618 thru 0x0b2a49 (0x3432 bytes) 000360[1] 1786352687.644289 mapread 0x071ef9 thru 0x07f88a (0xd992 bytes) 000362[0] 1786352687.681314 mapread 0x11ff32 thru 0x124fc6 (0x5095 bytes) 000363[1] 1786352687.708992 write 0x0d5f98 thru 0x0d6f54 (0xfbd bytes) 000365[0] 1786352687.787286 read 0x1bb336 thru 0x1c0c6f (0x593a bytes) 000367[0] 1786352687.850216 trunc from 0x1d5b5b to 0x56e95b (0x398e01 bytes) 000370[0] 1786352687.973966 trunc from 0x56e95c to 0x79c9f6 (0x22e09b bytes) 000373[0] 1786352688.108219 mapwrite 0x03c7b4 thru 0x040532 (0x3d7f bytes) 000377[0] 1786352688.338497 mapread 0x05f5bf thru 0x0633c6 (0x3e08 bytes) 000379[0] 1786352688.410330 write 0x13fc24 thru 0x14758d (0x796a bytes) 000382[0] 1786352688.465634 mapwrite 0x9ee7a2 thru 0x9ee953 (0x1b2 bytes) 000383[0] 1786352688.618453 write 0x89c8da thru 0x8a5930 (0x9057 bytes) 000385[1] 1786352688.691114 mapread 0x6b4a52 thru 0x6b758b (0x2b3a bytes) 000386[1] 1786352688.770298 mapread 0x2377e4 thru 0x24115a (0x9977 bytes) 000389[0] 1786352688.927031 mapread 0x4013e8 thru 0x407fb7 (0x6bd0 bytes) 000390[1] 1786352688.971316 trunc from 0x9ee954 to 0x41e998 (0x5cffbb bytes) 000391[1] 1786352689.054038 read 0x34e900 thru 0x355a77 (0x7178 bytes) 000392[0] 1786352689.065004 mapread 0x3b0677 thru 0x3ba8d5 (0xa25f bytes) 000393[1] 1786352689.177106 write 0x15e066 thru 0x161dd5 (0x3d70 bytes) 000395[0] 1786352689.234130 mapread 0x0fc0fa thru 0x1002af (0x41b6 bytes) 000398[1] 1786352689.337719 mapread 0x155817 thru 0x16077a (0xaf64 bytes) 000399[0] 1786352689.383147 mapwrite 0x307bf4 thru 0x31555e (0xd96b bytes) 000400[1] 1786352689.592960 write 0x340005 thru 0x34a4df (0xa4db bytes) 000401[0] 1786352689.625874 mapwrite 0x5f1175 thru 0x5fc341 (0xb1cd bytes) 000403[0] 1786352689.868746 mapwrite 0x26a4cd thru 0x26f7c7 (0x52fb bytes) 000405[0] 1786352690.077410 mapwrite 0x0ac6b4 thru 0x0af32a (0x2c77 bytes) 000409[1] 1786352690.192132 trunc from 0x5fc342 to 0x1c1be6 (0x43a75b bytes) 000410[1] 1786352690.243486 trunc from 0x1c1be7 to 0x155e58 (0x6bd8e bytes) 000412[0] 1786352690.314722 write 0x88a1ab thru 0x899edb (0xfd31 bytes) 000414[1] 1786352690.388600 read 0x20721f thru 0x20a972 (0x3754 bytes) 000415[1] 1786352690.447956 mapread 0x826b24 thru 0x82ab80 (0x405d bytes) 000418[1] 1786352690.518636 mapread 0x0a4faf thru 0x0a745c (0x24ae bytes) 000420[1] 1786352690.552607 trunc from 0x899edc to 0x289aea (0x6103f1 bytes) 000421[0] 1786352690.642230 mapwrite 0x983f21 thru 0x98c814 (0x88f4 bytes) 000422[0] 1786352690.836406 mapwrite 0x00047a thru 0x007356 (0x6edd bytes) 000424[1] 1786352690.928398 write 0x456672 thru 0x45aa8a (0x4419 bytes) 000426[0] 1786352690.981401 mapread 0x989472 thru 0x98c814 (0x33a3 bytes) 000427[0] 1786352691.011155 trunc from 0x98c815 to 0x085a8f (0x906d85 bytes) 000429[0] 1786352691.126400 write 0x9f5ef9 thru 0x9fffff (0xa107 bytes) 000431[0] 1786352691.170361 write 0x681ab7 thru 0x6898e1 (0x7e2b bytes) 000432[0] 1786352691.214457 write 0x626728 thru 0x6357e0 (0xf0b9 bytes) 000433[0] 1786352691.227415 mapread 0x842e9c thru 0x848264 (0x53c9 bytes) 000434[0] 1786352691.297253 write 0x01b0d6 thru 0x0226f3 (0x761e bytes) 000439[0] 1786352691.342054 mapread 0x6674cf thru 0x66e30d (0x6e3f bytes) 000444[1] 1786352691.433916 read 0x635d12 thru 0x642e12 (0xd101 bytes) 000446[0] 1786352691.483104 write 0x896ec8 thru 0x8a0b36 (0x9c6f bytes) 000448[0] 1786352691.523109 read 0x7bc70e thru 0x7c83b0 (0xbca3 bytes) 000452[1] 1786352691.609239 read 0x166e24 thru 0x1700ba (0x9297 bytes) 000453[1] 1786352691.662576 mapread 0x790eea thru 0x797a71 (0x6b88 bytes) 000454[1] 1786352691.707001 trunc from 0xa00000 to 0x46237c (0x59dc83 bytes) 000456[1] 1786352691.800981 trunc from 0x46237d to 0x55e1db (0xfbe5f bytes) 000457[1] 1786352691.833982 mapread 0x070c16 thru 0x07d92d (0xcd18 bytes) 000459[1] 1786352691.908572 write 0x92b686 thru 0x92f2e9 (0x3c64 bytes) 000461[1] 1786352691.952333 write 0x5ccaca thru 0x5ced1f (0x2256 bytes) 000462[0] 1786352691.966174 mapwrite 0x76f05c thru 0x778bd6 (0x9b7b bytes) 000463[0] 1786352692.325097 mapwrite 0x835df7 thru 0x83e1a0 (0x83aa bytes) 000464[1] 1786352692.466460 mapwrite 0x563420 thru 0x571751 (0xe332 bytes) 000465[0] 1786352692.653289 read 0x20482b thru 0x2140c5 (0xf89b bytes) 000466[1] 1786352692.697938 trunc from 0x92f2ea to 0x1be3da (0x770f0f bytes) 000468[0] 1786352692.789863 mapwrite 0x35a745 thru 0x3683ad (0xdc69 bytes) 000469[0] 1786352692.971900 read 0x04dc49 thru 0x04f636 (0x19ee bytes) 000471[1] 1786352693.068321 mapread 0x350418 thru 0x355f16 (0x5aff bytes) 000473[1] 1786352693.137163 mapwrite 0x83b4ec thru 0x846ac6 (0xb5db bytes) 000475[1] 1786352693.297096 mapread 0x649551 thru 0x64f5d4 (0x6084 bytes) 000485[1] 1786352693.507283 read 0x5d424c thru 0x5dccc1 (0x8a76 bytes) 000486[1] 1786352693.520319 mapwrite 0x043719 thru 0x04f13d (0xba25 bytes) 000487[0] 1786352693.611940 mapwrite 0x772e70 thru 0x77d0db (0xa26c bytes) 000488[0] 1786352693.798119 mapread 0x02a1bb thru 0x03419f (0x9fe5 bytes) 000490[0] 1786352693.914567 trunc from 0x846ac7 to 0x583958 (0x2c316e bytes) 000493[1] 1786352693.997642 mapread 0x0ec4de thru 0x0f3b14 (0x7637 bytes) 000494[1] 1786352694.070623 trunc from 0x583959 to 0x7c470c (0x240db4 bytes) 000496[0] 1786352694.152526 mapwrite 0x56861f thru 0x56cbc7 (0x45a9 bytes) 000497[0] 1786352694.436964 write 0x2ce565 thru 0x2dd2a6 (0xed42 bytes) 000499[0] 1786352694.538392 mapread 0x01ec2d thru 0x02488c (0x5c60 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 2694 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[0] 1786352695.523914 mapwrite 0x03d000 thru 0x042198 (0x5199 bytes) 000003[0] 1786352695.704346 write_OD 0x7d8000 thru 0x7e6fff (0xf000 bytes) 000004[0] 1786352695.820221 mapread 0x439000 thru 0x43d4cf (0x44d0 bytes) 000005[0] 1786352695.873075 read_OD 0x482000 thru 0x488fff (0x7000 bytes) 000008[1] 1786352695.886403 mapread 0x453000 thru 0x46235a (0xf35b bytes) 000009[1] 1786352695.918742 read 0x135000 thru 0x13a6b1 (0x56b2 bytes) truncating to largest ever: 0xe45c3 000012[1] 1786352695.936710 trunc from 0x7e7000 to 0x0e45c2 (0x702a3d bytes) 000013[1] 1786352696.000580 read 0x0c5000 thru 0x0cdfc0 (0x8fc1 bytes) 000014[0] 1786352696.056265 mapwrite 0x2a2000 thru 0x2a3265 (0x1266 bytes) 000015[0] 1786352696.189484 mapread 0x23d000 thru 0x247a84 (0xaa85 bytes) 000017[0] 1786352696.256587 mapread 0x068000 thru 0x06c067 (0x4068 bytes) 000019[1] 1786352696.344661 mapwrite 0x009000 thru 0x015915 (0xc916 bytes) 000021[0] 1786352696.607081 mapwrite 0x96e000 thru 0x972b73 (0x4b74 bytes) 000025[1] 1786352696.788069 mapread 0x50e000 thru 0x51ba06 (0xda07 bytes) 000028[0] 1786352696.923544 mapwrite 0x3e6000 thru 0x3ed2a2 (0x72a3 bytes) 000030[0] 1786352697.120319 mapwrite 0x93f000 thru 0x947658 (0x8659 bytes) 000031[1] 1786352697.260074 read 0x62c000 thru 0x630976 (0x4977 bytes) truncating to largest ever: 0x53c456 000035[0] 1786352697.285498 trunc from 0x972b74 to 0x53c455 (0x43671e bytes) 000036[1] 1786352697.313589 mapwrite 0x879000 thru 0x884621 (0xb622 bytes) 000039[1] 1786352697.477601 mapwrite 0x5ca000 thru 0x5d1b09 (0x7b0a bytes) truncating to largest ever: 0x883422 000043[1] 1786352697.604871 trunc from 0x884622 to 0x883421 (0x1200 bytes) 000044[0] 1786352697.633233 mapread 0x5a4000 thru 0x5a6038 (0x2039 bytes) 000047[1] 1786352697.693220 mapread 0x4c4000 thru 0x4cfe05 (0xbe06 bytes) 000048[1] 1786352697.729067 read 0x248000 thru 0x24add2 (0x2dd3 bytes) 000049[1] 1786352697.736086 read 0x848000 thru 0x8577da (0xf7db bytes) 000051[0] 1786352697.750663 mapread 0x32d000 thru 0x338251 (0xb252 bytes) 000053[0] 1786352697.793944 trunc from 0x883422 to 0x3d1230 (0x4b21f1 bytes) truncating to largest ever: 0x90a18d 000057[1] 1786352697.903460 trunc from 0x3d1231 to 0x90a18c (0x538f5c bytes) 000058[0] 1786352697.950008 mapwrite 0x63c000 thru 0x63f430 (0x3431 bytes) 000059[0] 1786352698.079298 write_OD 0x388000 thru 0x395fff (0xe000 bytes) 000061[0] 1786352698.156541 mapread 0x317000 thru 0x319294 (0x2295 bytes) 000064[1] 1786352698.189487 mapwrite 0x588000 thru 0x589791 (0x1792 bytes) 000067[1] 1786352698.310428 mapwrite 0x493000 thru 0x4a1fad (0xefae bytes) 000068[1] 1786352698.455184 mapread 0x533000 thru 0x53ecb6 (0xbcb7 bytes) 000069[0] 1786352698.494526 mapwrite 0x984000 thru 0x9934a4 (0xf4a5 bytes) 000071[0] 1786352698.691472 mapwrite 0x9c5000 thru 0x9ce340 (0x9341 bytes) 000072[1] 1786352698.817582 write 0x34f000 thru 0x3555b1 (0x65b2 bytes) 000073[0] 1786352698.858848 mapwrite 0x213000 thru 0x214fb9 (0x1fba bytes) 000075[1] 1786352698.944835 read 0x988000 thru 0x98ddf7 (0x5df8 bytes) 000078[0] 1786352698.972624 mapread 0x66f000 thru 0x67b4f9 (0xc4fa bytes) 000079[1] 1786352699.012269 read 0x58a000 thru 0x594fef (0xaff0 bytes) 000081[1] 1786352699.030443 write 0x3ee000 thru 0x3fd1dc (0xf1dd bytes) 000083[1] 1786352699.097596 read 0x8d4000 thru 0x8d5df8 (0x1df9 bytes) 000084[0] 1786352699.126199 mapread 0x6fd000 thru 0x709efe (0xceff bytes) 000086[1] 1786352699.204033 read 0x87e000 thru 0x887e6c (0x9e6d bytes) 000087[1] 1786352699.226258 trunc from 0x9ce341 to 0x1c43e2 (0x809f5e bytes) 000088[0] 1786352699.301234 mapread 0x13b000 thru 0x142b45 (0x7b46 bytes) 000089[1] 1786352699.355237 read 0x05e000 thru 0x068213 (0xa214 bytes) 000093[0] 1786352699.434947 mapwrite 0x6d9000 thru 0x6e8811 (0xf812 bytes) 000095[0] 1786352699.670477 mapwrite 0x41a000 thru 0x41f55b (0x555c bytes) 000097[0] 1786352699.783868 write_OD 0x388000 thru 0x391fff (0xa000 bytes) 000099[1] 1786352699.857837 read 0x2c6000 thru 0x2d051b (0xa51c bytes) 000101[1] 1786352699.879489 mapwrite 0x72c000 thru 0x72cb6a (0xb6b bytes) 000102[1] 1786352700.017668 write 0x518000 thru 0x5263d8 (0xe3d9 bytes) 000105[1] 1786352700.031594 trunc from 0x72cb6b to 0x114315 (0x618855 bytes) 000109[1] 1786352700.201560 write 0x93e000 thru 0x94da7a (0xfa7b bytes) 000112[1] 1786352700.242457 read 0x8e4000 thru 0x8f0370 (0xc371 bytes) 000113[1] 1786352700.301229 mapread 0x846000 thru 0x8500df (0xa0e0 bytes) 000116[1] 1786352700.406372 write 0x6c7000 thru 0x6d613b (0xf13c bytes) 000117[1] 1786352700.446524 mapread 0x6ad000 thru 0x6b44e8 (0x74e9 bytes) 000118[0] 1786352700.539523 mapread 0x60f000 thru 0x61a1bf (0xb1c0 bytes) 000122[1] 1786352700.705183 write 0x1ac000 thru 0x1b49ba (0x89bb bytes) 000123[0] 1786352700.772512 write_OD 0x813000 thru 0x819fff (0x7000 bytes) 000124[1] 1786352700.896293 mapwrite 0x8d7000 thru 0x8df18a (0x818b bytes) 000126[1] 1786352701.020465 mapwrite 0x314000 thru 0x31fddd (0xbdde bytes) 000127[0] 1786352701.155499 mapread 0x2d6000 thru 0x2de37c (0x837d bytes) 000128[1] 1786352701.208395 mapread 0x1e5000 thru 0x1e8952 (0x3953 bytes) 000129[0] 1786352701.241778 mapread 0x5a8000 thru 0x5b6676 (0xe677 bytes) 000130[1] 1786352701.276367 write 0x891000 thru 0x89713a (0x613b bytes) 000131[1] 1786352701.308383 write 0x11e000 thru 0x122658 (0x4659 bytes) 000132[1] 1786352701.333221 mapread 0x310000 thru 0x31af6e (0xaf6f bytes) 000133[1] 1786352701.369912 trunc from 0x94da7b to 0x2c95dd (0x68449d bytes) 000135[1] 1786352701.420781 mapwrite 0x323000 thru 0x32ed7e (0xbd7f bytes) 000136[1] 1786352701.584059 read 0x270000 thru 0x271ccd (0x1cce bytes) 000139[0] 1786352701.721542 mapwrite 0x294000 thru 0x294772 (0x773 bytes) 000140[0] 1786352701.823346 mapread 0x0b2000 thru 0x0c0dc9 (0xedca bytes) 000150[0] 1786352702.035284 trunc from 0x32ed7f to 0x829c5b (0x4faedd bytes) 000152[0] 1786352702.153508 write_OD 0x556000 thru 0x55dfff (0x8000 bytes) 000153[0] 1786352702.216801 write_OD 0x22e000 thru 0x234fff (0x7000 bytes) 000155[1] 1786352702.269332 mapwrite 0x30c000 thru 0x3123f2 (0x63f3 bytes) 000158[0] 1786352702.422042 read_OD 0x452000 thru 0x45efff (0xd000 bytes) 000159[1] 1786352702.456851 mapread 0x295000 thru 0x2995b6 (0x45b7 bytes) 000163[0] 1786352702.499361 mapwrite 0x324000 thru 0x32797b (0x397c bytes) 000164[0] 1786352702.586529 write_OD 0x40b000 thru 0x40efff (0x4000 bytes) 000165[0] 1786352702.652601 mapwrite 0x84c000 thru 0x857ecb (0xbecc bytes) 000166[0] 1786352702.811008 mapwrite 0x089000 thru 0x08d6ef (0x46f0 bytes) 000167[0] 1786352702.911520 trunc from 0x857ecc to 0x6f390a (0x1645c1 bytes) 000168[1] 1786352702.940929 mapwrite 0x9d8000 thru 0x9e32e0 (0xb2e1 bytes) 000169[0] 1786352703.182120 trunc from 0x9e32e1 to 0x7d970a (0x209bd6 bytes) 000172[0] 1786352703.284221 read_OD 0x611000 thru 0x61cfff (0xc000 bytes) 000174[1] 1786352703.316930 mapwrite 0x5b8000 thru 0x5c3179 (0xb17a bytes) 000175[1] 1786352703.473310 trunc from 0x7d970b to 0x845bbf (0x6c4b5 bytes) 000179[1] 1786352703.544461 write 0x38f000 thru 0x391451 (0x2452 bytes) 000183[1] 1786352703.575842 write 0x486000 thru 0x486e63 (0xe64 bytes) 000184[0] 1786352703.586095 read_OD 0x710000 thru 0x71afff (0xb000 bytes) 000187[1] 1786352703.645744 read 0x00d000 thru 0x0152c8 (0x82c9 bytes) 000189[0] 1786352703.668949 trunc from 0x845bc0 to 0x20f795 (0x63642a bytes) 000190[0] 1786352703.745321 write_OD 0x5c6000 thru 0x5d1fff (0xc000 bytes) 000193[1] 1786352703.840944 read 0x12d000 thru 0x12d244 (0x245 bytes) 000194[1] 1786352703.849313 mapwrite 0x15f000 thru 0x165d15 (0x6d16 bytes) 000199[1] 1786352703.995504 mapread 0x483000 thru 0x48309c (0x9d bytes) 000200[0] 1786352704.030786 write_OD 0x8ce000 thru 0x8d1fff (0x4000 bytes) 000201[1] 1786352704.120969 read 0x363000 thru 0x369079 (0x607a bytes) 000205[0] 1786352704.150782 trunc from 0x8d2000 to 0x0c03fc (0x811c03 bytes) 000206[1] 1786352704.250582 write 0x447000 thru 0x44fcaa (0x8cab bytes) 000208[0] 1786352704.273823 trunc from 0x44fcab to 0x4429e6 (0xd2c4 bytes) 000209[0] 1786352704.359258 write_OD 0x413000 thru 0x416fff (0x4000 bytes) 000215[1] 1786352704.443038 mapread 0x1ce000 thru 0x1d77ec (0x97ed bytes) 000219[0] 1786352704.471301 trunc from 0x4429e7 to 0x9031e3 (0x4c07fd bytes) 000225[1] 1786352704.603922 mapread 0x6f5000 thru 0x6f805e (0x305f bytes) 000226[0] 1786352704.669394 write_OD 0x252000 thru 0x258fff (0x7000 bytes) 000228[0] 1786352704.742327 mapread 0x11e000 thru 0x11ecbb (0xcbc bytes) 000229[1] 1786352704.749879 mapread 0x269000 thru 0x273f7e (0xaf7f bytes) 000230[0] 1786352704.779288 trunc from 0x9031e4 to 0x6d6e40 (0x22c3a3 bytes) 000232[1] 1786352704.857406 mapread 0x1bd000 thru 0x1be4ca (0x14cb bytes) 000233[0] 1786352704.887616 mapread 0x04e000 thru 0x05dca3 (0xfca4 bytes) 000234[0] 1786352704.917495 write_OD 0x17a000 thru 0x184fff (0xb000 bytes) 000235[1] 1786352704.994067 mapread 0x571000 thru 0x576920 (0x5921 bytes) 000237[1] 1786352705.051576 trunc from 0x6d6e41 to 0x7307ff (0x599bf bytes) 000238[1] 1786352705.146011 mapwrite 0x2ae000 thru 0x2b58f0 (0x78f1 bytes) 000240[0] 1786352705.246303 mapread 0x3cc000 thru 0x3da58c (0xe58d bytes) 000247[1] 1786352705.326515 read 0x262000 thru 0x26504d (0x304e bytes) 000248[1] 1786352705.334505 mapread 0x679000 thru 0x6826d3 (0x96d4 bytes) 000249[1] 1786352705.355897 write 0x402000 thru 0x40ed57 (0xcd58 bytes) 000251[1] 1786352705.425632 trunc from 0x730800 to 0x80a1d9 (0xd99da bytes) 000253[0] 1786352705.453375 write_OD 0x500000 thru 0x505fff (0x6000 bytes) 000254[1] 1786352705.515519 mapwrite 0x712000 thru 0x714888 (0x2889 bytes) 000255[0] 1786352705.586591 write_OD 0x8ba000 thru 0x8c5fff (0xc000 bytes) 000256[0] 1786352705.646956 write_OD 0x4aa000 thru 0x4b5fff (0xc000 bytes) 000258[1] 1786352705.731504 read 0x562000 thru 0x568ebe (0x6ebf bytes) 000259[1] 1786352705.750394 read 0x1d8000 thru 0x1e15a5 (0x95a6 bytes) 000261[1] 1786352705.761567 trunc from 0x8c6000 to 0x135cc7 (0x790338 bytes) 000262[0] 1786352705.831291 mapwrite 0x5f6000 thru 0x5f8941 (0x2942 bytes) 000266[1] 1786352705.978497 write 0x661000 thru 0x66e223 (0xd224 bytes) 000268[1] 1786352705.997108 trunc from 0x66e224 to 0x5aa4fb (0xc3d28 bytes) 000269[1] 1786352706.028865 trunc from 0x5aa4fc to 0x6281e4 (0x7dce9 bytes) 000271[0] 1786352706.071598 trunc from 0x6281e5 to 0x77aab2 (0x1528ce bytes) 000273[0] 1786352706.133342 read_OD 0x282000 thru 0x284fff (0x3000 bytes) 000276[0] 1786352706.152176 trunc from 0x77aab3 to 0x89b5a1 (0x120aef bytes) 000278[0] 1786352706.207125 mapread 0x0a0000 thru 0x0a5229 (0x522a bytes) 000279[0] 1786352706.255424 write_OD 0x992000 thru 0x9a0fff (0xf000 bytes) 000280[1] 1786352706.333228 mapwrite 0x769000 thru 0x76bcd8 (0x2cd9 bytes) 000282[1] 1786352706.463475 mapread 0x2c0000 thru 0x2cea99 (0xea9a bytes) 000283[1] 1786352706.525318 trunc from 0x9a1000 to 0x80afe6 (0x196019 bytes) 000285[1] 1786352706.578262 write 0x1c7000 thru 0x1c8eb4 (0x1eb5 bytes) 000286[0] 1786352706.585914 mapread 0x470000 thru 0x47e587 (0xe588 bytes) 000287[1] 1786352706.664471 trunc from 0x80afe7 to 0x83ec61 (0x33c7b bytes) 000293[0] 1786352706.807579 write_OD 0x4e8000 thru 0x4eefff (0x7000 bytes) 000297[1] 1786352706.874070 trunc from 0x83ec62 to 0x803caf (0x3afb2 bytes) 000298[1] 1786352706.936065 write 0x039000 thru 0x043e19 (0xae1a bytes) 000301[0] 1786352706.989504 trunc from 0x803cb0 to 0x0703b7 (0x7938f8 bytes) 000307[1] 1786352707.087486 read 0x02e000 thru 0x03c0fc (0xe0fd bytes) 000308[1] 1786352707.126196 read 0x027000 thru 0x02a8d4 (0x38d5 bytes) 000309[1] 1786352707.126829 write 0x702000 thru 0x706c12 (0x4c13 bytes) 000311[1] 1786352707.208652 write 0x0b6000 thru 0x0bd187 (0x7188 bytes) 000316[0] 1786352707.347250 read_OD 0x058000 thru 0x059fff (0x2000 bytes) 000317[1] 1786352707.418587 mapread 0x22b000 thru 0x22d161 (0x2162 bytes) 000318[0] 1786352707.523043 read_OD 0x238000 thru 0x244fff (0xd000 bytes) 000322[0] 1786352707.537612 read_OD 0x049000 thru 0x057fff (0xf000 bytes) 000323[0] 1786352707.553883 write_OD 0x019000 thru 0x01cfff (0x4000 bytes) 000325[1] 1786352707.631322 trunc from 0x706c13 to 0x4e395d (0x2232b5 bytes) 000326[1] 1786352707.674221 mapread 0x13e000 thru 0x149bf7 (0xbbf8 bytes) 000327[1] 1786352707.776273 mapread 0x2ca000 thru 0x2d9f8b (0xff8c bytes) 000331[1] 1786352707.897779 trunc from 0x4e395e to 0x01dffa (0x4c5963 bytes) 000333[0] 1786352707.945127 read_OD 0x017000 thru 0x01cfff (0x6000 bytes) 000335[1] 1786352707.991888 mapread 0x01a000 thru 0x01d34c (0x334d bytes) 000337[1] 1786352708.029293 mapread 0x00d000 thru 0x01a159 (0xd15a bytes) 000339[0] 1786352708.117892 read_OD 0x008000 thru 0x013fff (0xc000 bytes) 000340[0] 1786352708.133473 mapwrite 0x2d1000 thru 0x2d4144 (0x3145 bytes) 000343[0] 1786352708.344530 mapread 0x020000 thru 0x02041f (0x420 bytes) 000344[0] 1786352708.370237 mapread 0x0cb000 thru 0x0d20dc (0x70dd bytes) 000345[1] 1786352708.389485 mapread 0x01b000 thru 0x0298fb (0xe8fc bytes) 000350[1] 1786352708.524628 read 0x1fb000 thru 0x1fd86d (0x286e bytes) 000353[0] 1786352708.605488 read_OD 0x1bf000 thru 0x1cbfff (0xd000 bytes) 000356[0] 1786352708.621553 mapread 0x1ac000 thru 0x1b77eb (0xb7ec bytes) 000357[1] 1786352708.721015 mapread 0x238000 thru 0x23d4d2 (0x54d3 bytes) 000359[1] 1786352708.722393 write 0x15a000 thru 0x15d2f7 (0x32f8 bytes) 000363[1] 1786352708.857570 mapread 0x1aa000 thru 0x1ac795 (0x2796 bytes) 000365[1] 1786352708.931226 mapwrite 0x944000 thru 0x953638 (0xf639 bytes) 000366[0] 1786352709.223393 trunc from 0x953639 to 0x41df5a (0x5356de bytes) 000367[0] 1786352709.292012 mapread 0x11b000 thru 0x11b5f4 (0x5f5 bytes) 000369[0] 1786352709.386563 read_OD 0x32e000 thru 0x337fff (0xa000 bytes) 000372[1] 1786352709.428602 trunc from 0x41df5b to 0x16faed (0x2ae46d bytes) 000374[1] 1786352709.517196 read 0x01d000 thru 0x028932 (0xb933 bytes) 000376[0] 1786352709.585306 write_OD 0x7e6000 thru 0x7ecfff (0x7000 bytes) 000380[0] 1786352709.676626 write_OD 0x7ac000 thru 0x7affff (0x4000 bytes) 000381[1] 1786352709.757724 read 0x197000 thru 0x19fefa (0x8efb bytes) 000382[1] 1786352709.771968 mapread 0x1f6000 thru 0x1ffb03 (0x9b04 bytes) 000385[0] 1786352709.804625 trunc from 0x7ed000 to 0x06b652 (0x7819ad bytes) 000387[1] 1786352709.884848 trunc from 0x06b653 to 0x837688 (0x7cc036 bytes) 000389[0] 1786352709.961254 read_OD 0x279000 thru 0x27dfff (0x5000 bytes) truncating to largest ever: 0x9fafea 000392[0] 1786352710.003998 trunc from 0x837689 to 0x9fafe9 (0x1c3961 bytes) 000393[1] 1786352710.067236 trunc from 0x9fafea to 0x15977e (0x8a186b bytes) 000394[1] 1786352710.109442 read 0x11b000 thru 0x11cb3f (0x1b40 bytes) 000396[1] 1786352710.173361 write 0x89a000 thru 0x8a0c59 (0x6c5a bytes) 000397[0] 1786352710.183856 read_OD 0x712000 thru 0x71dfff (0xc000 bytes) 000400[0] 1786352710.247544 write_OD 0x77d000 thru 0x77ffff (0x3000 bytes) 000402[0] 1786352710.316383 mapread 0x281000 thru 0x28598b (0x498c bytes) 000405[0] 1786352710.393485 mapwrite 0x0f6000 thru 0x10584c (0xf84d bytes) 000406[0] 1786352710.708232 mapread 0x17d000 thru 0x180826 (0x3827 bytes) 000407[1] 1786352710.764998 mapread 0x713000 thru 0x71bb18 (0x8b19 bytes) 000408[0] 1786352710.973362 mapwrite 0x4e4000 thru 0x4ecef7 (0x8ef8 bytes) 000410[0] 1786352711.200116 mapread 0x2b5000 thru 0x2c02e9 (0xb2ea bytes) 000415[0] 1786352711.303590 mapwrite 0x526000 thru 0x52d571 (0x7572 bytes) 000417[1] 1786352711.377274 read 0x330000 thru 0x3367b2 (0x67b3 bytes) 000421[1] 1786352711.415367 read 0x5ae000 thru 0x5b1472 (0x3473 bytes) 000422[1] 1786352711.426969 write 0x7fd000 thru 0x801b6d (0x4b6e bytes) 000426[0] 1786352711.554106 trunc from 0x8a0c5a to 0x9783cf (0xd7776 bytes) 000429[1] 1786352711.650630 write 0x7ed000 thru 0x7f9a26 (0xca27 bytes) 000430[0] 1786352711.672386 read_OD 0x6b8000 thru 0x6c4fff (0xd000 bytes) 000431[1] 1786352711.732306 write 0x5ac000 thru 0x5b931f (0xd320 bytes) 000433[0] 1786352711.796938 read_OD 0x412000 thru 0x41dfff (0xc000 bytes) 000435[0] 1786352711.846542 mapwrite 0x22c000 thru 0x2335eb (0x75ec bytes) 000436[1] 1786352711.954996 mapwrite 0x7fd000 thru 0x807567 (0xa568 bytes) 000438[0] 1786352712.094305 mapwrite 0x8ea000 thru 0x8ea25f (0x260 bytes) 000439[1] 1786352712.166435 trunc from 0x9783d0 to 0x9bfd9d (0x479ce bytes) 000445[1] 1786352712.235138 mapread 0x059000 thru 0x068701 (0xf702 bytes) 000446[0] 1786352712.267413 trunc from 0x9bfd9e to 0x367837 (0x658566 bytes) 000447[0] 1786352712.308252 read_OD 0x0c2000 thru 0x0cffff (0xe000 bytes) 000448[1] 1786352712.333636 mapread 0x188000 thru 0x1975b9 (0xf5ba bytes) 000450[1] 1786352712.379235 mapread 0x29c000 thru 0x2a950d (0xd50e bytes) 000453[0] 1786352712.429533 mapread 0x2ad000 thru 0x2b34cc (0x64cd bytes) 000455[1] 1786352712.537472 mapwrite 0x6dc000 thru 0x6eb877 (0xf878 bytes) 000456[1] 1786352712.697509 mapread 0x4a2000 thru 0x4a3fbc (0x1fbd bytes) 000457[0] 1786352712.718017 read_OD 0x47e000 thru 0x485fff (0x8000 bytes) 000458[0] 1786352712.751533 mapwrite 0x0b5000 thru 0x0b737c (0x237d bytes) 000460[1] 1786352712.887308 mapread 0x4ec000 thru 0x4f5f58 (0x9f59 bytes) 000461[0] 1786352712.925475 write_OD 0x1ec000 thru 0x1f6fff (0xb000 bytes) 000462[0] 1786352712.964533 read_OD 0x5a0000 thru 0x5a7fff (0x8000 bytes) 000463[1] 1786352712.981570 write 0x5ea000 thru 0x5f9ab5 (0xfab6 bytes) 000464[1] 1786352713.028003 mapwrite 0x5cc000 thru 0x5d3972 (0x7973 bytes) 000466[0] 1786352713.106165 mapwrite 0x187000 thru 0x192f24 (0xbf25 bytes) 000469[0] 1786352713.257874 trunc from 0x6eb878 to 0x426548 (0x2c532f bytes) 000470[1] 1786352713.304618 mapwrite 0x196000 thru 0x196992 (0x993 bytes) 000473[1] 1786352713.418581 write 0x580000 thru 0x589e6a (0x9e6b bytes) 000474[0] 1786352713.437587 mapwrite 0x3c6000 thru 0x3d121b (0xb21c bytes) 000475[1] 1786352713.580929 write 0x812000 thru 0x81bee4 (0x9ee5 bytes) 000480[1] 1786352713.651449 mapwrite 0x264000 thru 0x264f07 (0xf08 bytes) 000483[0] 1786352713.717340 mapwrite 0x50f000 thru 0x51a6fb (0xb6fc bytes) 000485[1] 1786352713.874075 read 0x2f0000 thru 0x2ff3bb (0xf3bc bytes) 000486[0] 1786352713.921144 mapwrite 0x7f0000 thru 0x7fc06d (0xc06e bytes) 000487[0] 1786352714.040488 mapread 0x625000 thru 0x62c8ff (0x7900 bytes) 000489[1] 1786352714.078942 trunc from 0x81bee5 to 0x7f5844 (0x266a0 bytes) 000491[0] 1786352714.136561 trunc from 0x7f5845 to 0x6c0919 (0x134f2b bytes) 000492[0] 1786352714.184637 mapwrite 0x08b000 thru 0x09572d (0xa72e bytes) 000493[1] 1786352714.263650 write 0x5e7000 thru 0x5ef5b6 (0x85b7 bytes) 000498[1] 1786352714.340846 write 0x956000 thru 0x95e779 (0x877a bytes) All operations completed A-OK! PASS 16a (53s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 05:05:19 (1786352719) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.382121 s, 22.0 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.378997 s, 22.1 MB/s Chance of close/open is 1 in 50 Seed set to 2722 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read skipping zero size read skipping zero size read skipping zero size read skipping zero size read skipping zero size read 000026[1] 1786352723.504405 write 0x3ff02e thru 0x40000c (0xfdf bytes) 000027[0] 1786352723.539264 read 0x29e525 thru 0x29fafa (0x15d6 bytes) 000028[1] 1786352723.584240 write 0x215d70 thru 0x2164cb (0x75c bytes) truncating to largest ever: 0x1e076b 000029[0] 1786352723.608968 trunc from 0x40000d to 0x1e076a (0x21f8a2 bytes) 000030[1] 1786352723.695888 mapwrite 0x6548dd thru 0x6552f5 (0xa19 bytes) 000031[0] 1786352723.841120 mapwrite 0x2b3fc7 thru 0x2b40bb (0xf5 bytes) 000032[1] 1786352723.948949 mapread 0x03b396 thru 0x03beb9 (0xb24 bytes) 000034[1] 1786352723.984143 read 0x182174 thru 0x182bc0 (0xa4d bytes) 000036[0] 1786352724.033025 mapread 0x475162 thru 0x476b51 (0x19f0 bytes) 000037[1] 1786352724.071541 read 0x638956 thru 0x639d95 (0x1440 bytes) 000043[0] 1786352724.081904 read 0x244592 thru 0x244b13 (0x582 bytes) truncating to largest ever: 0x9bfd48 000050[0] 1786352724.094220 trunc from 0x6552f6 to 0x9bfd47 (0x36aa52 bytes) 000056[0] 1786352724.272051 mapread 0x4d8903 thru 0x4d9bae (0x12ac bytes) 000057[0] 1786352724.315844 mapwrite 0x67e2e7 thru 0x67f243 (0xf5d bytes) 000058[0] 1786352724.393472 mapwrite 0x1b26c1 thru 0x1b40b0 (0x19f0 bytes) 000060[0] 1786352724.502860 read 0x4cb648 thru 0x4cc2c3 (0xc7c bytes) 000063[0] 1786352724.555320 write 0x79ae28 thru 0x79b21c (0x3f5 bytes) 000067[1] 1786352724.595064 mapread 0x4be4fa thru 0x4bea5f (0x566 bytes) 000068[0] 1786352724.664430 trunc from 0x9bfd48 to 0x06451e (0x95b829 bytes) 000071[0] 1786352724.813404 write 0x2d979a thru 0x2db1ce (0x1a35 bytes) 000072[0] 1786352724.851374 mapwrite 0x78ce51 thru 0x78eb14 (0x1cc4 bytes) 000073[0] 1786352725.031350 read 0x3d2017 thru 0x3d2a2b (0xa15 bytes) 000074[1] 1786352725.051365 trunc from 0x78eb15 to 0x8a0df4 (0x1122e0 bytes) 000076[0] 1786352725.119094 read 0x029976 thru 0x02b7a1 (0x1e2c bytes) 000077[0] 1786352725.150399 mapwrite 0x46f42c thru 0x47021f (0xdf4 bytes) 000080[0] 1786352725.280443 read 0x88bf2a thru 0x88d88f (0x1966 bytes) 000082[1] 1786352725.309262 write 0x5c3635 thru 0x5c4201 (0xbcd bytes) 000083[1] 1786352725.365628 write 0x4372b5 thru 0x437b15 (0x861 bytes) 000085[0] 1786352725.375481 mapwrite 0x3c1b88 thru 0x3c23aa (0x823 bytes) 000086[1] 1786352725.487019 trunc from 0x8a0df5 to 0x1be7fa (0x6e25fa bytes) 000089[0] 1786352725.653032 read 0x165314 thru 0x165d4b (0xa38 bytes) 000091[1] 1786352725.732139 mapread 0x0a00b2 thru 0x0a1885 (0x17d4 bytes) 000093[1] 1786352725.746213 trunc from 0x1be7fb to 0x64ad93 (0x48c599 bytes) 000094[0] 1786352725.847851 write 0x4e3617 thru 0x4e3c67 (0x651 bytes) 000095[1] 1786352725.887118 write 0x3c3ef0 thru 0x3c555e (0x166f bytes) 000099[0] 1786352725.987025 mapwrite 0x62f2df thru 0x62ff04 (0xc26 bytes) 000100[0] 1786352726.064042 mapwrite 0x49b2b0 thru 0x49be52 (0xba3 bytes) 000102[0] 1786352726.164372 write 0x301e58 thru 0x302014 (0x1bd bytes) 000104[0] 1786352726.332657 read 0x5010d7 thru 0x5025d9 (0x1503 bytes) 000107[0] 1786352726.432128 mapwrite 0x78003c thru 0x781133 (0x10f8 bytes) 000108[1] 1786352726.552968 write 0x520179 thru 0x520fb9 (0xe41 bytes) 000109[0] 1786352726.604216 write 0x0fb71b thru 0x0fc92c (0x1212 bytes) 000112[0] 1786352726.686945 mapwrite 0x2cb708 thru 0x2ccf51 (0x184a bytes) 000115[0] 1786352726.826301 trunc from 0x781134 to 0x6ad94d (0xd37e6 bytes) 000116[1] 1786352726.909804 mapread 0x1d6dca thru 0x1d754d (0x784 bytes) 000119[1] 1786352727.018902 read 0x186ff2 thru 0x1877ce (0x7dd bytes) 000122[1] 1786352727.028109 mapwrite 0x3da7b0 thru 0x3daeb4 (0x705 bytes) 000123[0] 1786352727.152561 write 0x0bf9bb thru 0x0c1617 (0x1c5d bytes) 000125[0] 1786352727.209371 mapwrite 0x1d81fc thru 0x1d883c (0x641 bytes) 000126[0] 1786352727.291660 write 0x7d0eb1 thru 0x7d14f9 (0x649 bytes) 000128[0] 1786352727.294939 trunc from 0x7d14fa to 0x55b16f (0x27638a bytes) 000131[0] 1786352727.380478 write 0x8490de thru 0x84abe5 (0x1b08 bytes) 000132[0] 1786352727.409410 mapread 0x45eda4 thru 0x45f89f (0xafc bytes) 000135[1] 1786352727.456048 mapread 0x260463 thru 0x2623bc (0x1f5a bytes) 000139[0] 1786352727.547386 mapwrite 0x283a89 thru 0x284013 (0x58b bytes) 000140[0] 1786352727.644937 trunc from 0x84abe6 to 0x273dc9 (0x5d6e1c bytes) 000141[1] 1786352727.676310 mapwrite 0x8d8b39 thru 0x8d91a9 (0x671 bytes) 000143[0] 1786352727.817839 mapwrite 0x167b85 thru 0x168622 (0xa9e bytes) 000145[0] 1786352727.940574 write 0x523532 thru 0x525521 (0x1ff0 bytes) 000147[0] 1786352728.014809 read 0x719cd2 thru 0x71b490 (0x17bf bytes) 000151[1] 1786352728.084551 read 0x48aadd thru 0x48cabc (0x1fe0 bytes) 000153[1] 1786352728.150068 trunc from 0x8d91aa to 0x03682f (0x8a297a bytes) 000157[1] 1786352728.303906 mapwrite 0x89aaf7 thru 0x89ca79 (0x1f83 bytes) 000158[1] 1786352728.428622 trunc from 0x89ca7a to 0x7f0311 (0xac768 bytes) 000159[0] 1786352728.457410 mapwrite 0x126cca thru 0x128a5a (0x1d91 bytes) 000160[1] 1786352728.561621 mapread 0x13fed8 thru 0x140fe2 (0x110b bytes) 000164[0] 1786352728.620033 mapread 0x25173c thru 0x252408 (0xccd bytes) 000165[0] 1786352728.654247 mapread 0x5b0317 thru 0x5b1e4f (0x1b39 bytes) 000168[0] 1786352728.707852 read 0x0f662c thru 0x0f6781 (0x156 bytes) 000169[1] 1786352728.715193 write 0x3da90a thru 0x3db53e (0xc35 bytes) 000170[1] 1786352728.769191 mapwrite 0x10d787 thru 0x10d835 (0xaf bytes) 000174[0] 1786352728.918653 trunc from 0x7f0312 to 0x1a413e (0x64c1d3 bytes) 000176[0] 1786352729.016866 write 0x5234bb thru 0x524671 (0x11b7 bytes) 000181[0] 1786352729.054176 trunc from 0x524672 to 0x1fd3de (0x327293 bytes) 000182[0] 1786352729.108911 mapwrite 0x84d40b thru 0x84e04c (0xc42 bytes) 000183[1] 1786352729.218963 mapwrite 0x8c0f0d thru 0x8c1439 (0x52d bytes) 000186[1] 1786352729.361988 write 0x68782d thru 0x6891f2 (0x19c6 bytes) 000187[1] 1786352729.383335 mapwrite 0x36dcbf thru 0x36ee29 (0x116b bytes) 000188[1] 1786352729.494932 mapwrite 0x9338a4 thru 0x934034 (0x791 bytes) 000191[1] 1786352729.671971 write 0x96d3b0 thru 0x96ebe5 (0x1836 bytes) 000193[1] 1786352729.707867 write 0x692bcd thru 0x694181 (0x15b5 bytes) 000195[0] 1786352729.746326 read 0x5968bd thru 0x5977bd (0xf01 bytes) 000197[1] 1786352729.810601 write 0x880ec3 thru 0x88107f (0x1bd bytes) 000198[1] 1786352729.837029 mapread 0x636cb9 thru 0x6370c6 (0x40e bytes) 000200[0] 1786352729.914864 write 0x273422 thru 0x27395a (0x539 bytes) 000203[0] 1786352729.963616 mapread 0x7386e4 thru 0x739c27 (0x1544 bytes) 000205[0] 1786352730.007074 read 0x083648 thru 0x085498 (0x1e51 bytes) 000206[0] 1786352730.021322 write 0x9b8369 thru 0x9ba040 (0x1cd8 bytes) 000207[0] 1786352730.050064 read 0x79bc16 thru 0x79ccaa (0x1095 bytes) 000209[1] 1786352730.104247 mapread 0x5be4de thru 0x5bef1a (0xa3d bytes) 000212[0] 1786352730.142101 trunc from 0x9ba041 to 0x22a7f0 (0x78f850 bytes) 000213[1] 1786352730.209546 read 0x1d3d11 thru 0x1d3f0f (0x1ff bytes) 000216[0] 1786352730.332256 mapwrite 0x2f0a9e thru 0x2f1dda (0x133d bytes) 000217[0] 1786352730.479050 read 0x05c2f1 thru 0x05c59c (0x2ac bytes) 000222[0] 1786352730.562904 trunc from 0x2f1ddb to 0x46f608 (0x17d82e bytes) 000224[0] 1786352730.642158 read 0x1d5bc9 thru 0x1d6ef4 (0x132c bytes) 000226[1] 1786352730.648033 write 0x622c68 thru 0x6243b4 (0x174d bytes) 000227[0] 1786352730.689420 write 0x572634 thru 0x572fb4 (0x981 bytes) 000228[1] 1786352730.752615 mapread 0x3a3cd8 thru 0x3a5268 (0x1591 bytes) 000229[1] 1786352730.858181 read 0x0f535b thru 0x0f5e5d (0xb03 bytes) 000230[0] 1786352730.924571 mapwrite 0x987aac thru 0x989903 (0x1e58 bytes) 000232[0] 1786352731.119125 mapread 0x1bb6ca thru 0x1bd04b (0x1982 bytes) 000233[0] 1786352731.218887 mapread 0x32c93a thru 0x32d103 (0x7ca bytes) 000234[0] 1786352731.238796 mapwrite 0x3ae99c thru 0x3aeacd (0x132 bytes) 000236[0] 1786352731.375340 write 0x824819 thru 0x8260bb (0x18a3 bytes) 000237[1] 1786352731.405326 mapread 0x7ba2f0 thru 0x7bbaec (0x17fd bytes) 000240[1] 1786352731.534509 read 0x339f9d thru 0x33b850 (0x18b4 bytes) 000242[1] 1786352731.571909 trunc from 0x989904 to 0x3fcfe4 (0x58c91f bytes) 000245[1] 1786352731.731924 mapread 0x10d2e4 thru 0x10d6dd (0x3fa bytes) 000247[1] 1786352731.746397 mapwrite 0x97c9eb thru 0x97cc1f (0x235 bytes) 000248[0] 1786352731.925613 read 0x080b54 thru 0x081e84 (0x1331 bytes) 000249[1] 1786352731.982631 mapwrite 0x40d997 thru 0x40daee (0x158 bytes) 000250[1] 1786352732.061767 mapread 0x099fd4 thru 0x09bcd2 (0x1cff bytes) 000251[0] 1786352732.100655 trunc from 0x97cc20 to 0x19681e (0x7e6401 bytes) 000252[1] 1786352732.157322 mapread 0x02af8c thru 0x02ca11 (0x1a86 bytes) 000254[1] 1786352732.203139 write 0x635949 thru 0x637151 (0x1809 bytes) truncating to largest ever: 0x9ffcc0 000255[1] 1786352732.240574 trunc from 0x637152 to 0x9ffcbf (0x3c8b6e bytes) 000256[0] 1786352732.273165 read 0x5b0728 thru 0x5b13c0 (0xc99 bytes) 000257[0] 1786352732.300544 read 0x4eae32 thru 0x4eaff5 (0x1c4 bytes) 000260[0] 1786352732.352832 mapwrite 0x689f6e thru 0x68b401 (0x1494 bytes) 000261[1] 1786352732.468040 mapwrite 0x61a14a thru 0x61b32b (0x11e2 bytes) 000263[0] 1786352732.614909 write 0x76c49b thru 0x76df79 (0x1adf bytes) 000264[0] 1786352732.659811 trunc from 0x9ffcc0 to 0x7226ab (0x2dd614 bytes) 000267[1] 1786352732.770390 read 0x639d82 thru 0x63b104 (0x1383 bytes) 000273[1] 1786352732.862796 trunc from 0x7226ac to 0x89f234 (0x17cb89 bytes) 000275[0] 1786352732.950061 write 0x3aa058 thru 0x3aa4c5 (0x46e bytes) 000276[0] 1786352732.996943 mapread 0x75086f thru 0x752520 (0x1cb2 bytes) 000278[1] 1786352733.090753 write 0x3e0027 thru 0x3e1025 (0xfff bytes) 000280[0] 1786352733.139032 mapwrite 0x90adc5 thru 0x90b6a5 (0x8e1 bytes) 000282[1] 1786352733.305923 write 0x81d65d thru 0x81edbc (0x1760 bytes) 000283[1] 1786352733.349080 mapread 0x6fa4cc thru 0x6fbce0 (0x1815 bytes) 000285[0] 1786352733.420570 mapread 0x340071 thru 0x3415d9 (0x1569 bytes) 000287[1] 1786352733.468914 mapread 0x5c74f6 thru 0x5c8c0b (0x1716 bytes) 000289[1] 1786352733.481516 mapread 0x88bc71 thru 0x88d257 (0x15e7 bytes) 000291[1] 1786352733.502130 trunc from 0x90b6a6 to 0x6e9a12 (0x221c93 bytes) 000294[1] 1786352733.610557 read 0x4e0db3 thru 0x4e1968 (0xbb6 bytes) 000299[1] 1786352733.685285 mapread 0x117dba thru 0x118119 (0x360 bytes) 000302[0] 1786352733.706205 trunc from 0x6e9a13 to 0x4304ce (0x2b9544 bytes) 000304[1] 1786352733.770475 mapread 0x12afd7 thru 0x12c30a (0x1334 bytes) 000307[1] 1786352733.850110 read 0x0187f0 thru 0x018b90 (0x3a1 bytes) 000310[1] 1786352733.882311 mapwrite 0x1b62c9 thru 0x1b81f2 (0x1f2a bytes) 000314[1] 1786352734.063390 mapread 0x1fb715 thru 0x1fc50b (0xdf7 bytes) 000315[0] 1786352734.107543 write 0x4d67b5 thru 0x4d84de (0x1d2a bytes) 000316[1] 1786352734.134479 mapread 0x45a3cc thru 0x45b8de (0x1513 bytes) 000317[1] 1786352734.182467 write 0x34ccb8 thru 0x34d3fd (0x746 bytes) 000318[1] 1786352734.225908 trunc from 0x4d84df to 0x3b0a1f (0x127abf bytes) 000320[0] 1786352734.289426 trunc from 0x3b0a20 to 0x6c751b (0x316afc bytes) 000324[1] 1786352734.461330 read 0x4bf1c0 thru 0x4c0ac4 (0x1905 bytes) 000325[0] 1786352734.512101 mapwrite 0x35ba97 thru 0x35bfde (0x548 bytes) 000327[1] 1786352734.620626 trunc from 0x6c751c to 0x1293a5 (0x59e176 bytes) 000335[0] 1786352734.888070 mapread 0x05b0db thru 0x05b65f (0x585 bytes) 000336[1] 1786352734.941812 trunc from 0x1293a6 to 0x3665c4 (0x23d21f bytes) 000338[1] 1786352735.040477 read 0x2f7386 thru 0x2f7fb2 (0xc2d bytes) 000339[0] 1786352735.172443 mapread 0x0e8c1e thru 0x0eaabe (0x1ea1 bytes) 000340[0] 1786352735.227833 mapwrite 0x2b7bd5 thru 0x2b98b9 (0x1ce5 bytes) 000343[1] 1786352735.398592 write 0x7d858e thru 0x7d8780 (0x1f3 bytes) 000347[1] 1786352735.475323 mapwrite 0x0970f2 thru 0x097ffb (0xf0a bytes) 000352[1] 1786352735.637429 mapread 0x454cb6 thru 0x455eb5 (0x1200 bytes) 000353[0] 1786352735.647914 read 0x57538b thru 0x575a36 (0x6ac bytes) 000355[0] 1786352735.703762 mapread 0x4157bd thru 0x415cb2 (0x4f6 bytes) 000357[0] 1786352735.776174 trunc from 0x7d8781 to 0x4c3995 (0x314deb bytes) 000359[1] 1786352735.917807 mapwrite 0x69ff5c thru 0x6a173e (0x17e3 bytes) 000360[1] 1786352736.159594 mapread 0x0220c6 thru 0x023930 (0x186b bytes) 000365[1] 1786352736.236267 read 0x5ffa98 thru 0x5ffe98 (0x401 bytes) 000366[1] 1786352736.237908 read 0x50de28 thru 0x50ee1d (0xff6 bytes) 000368[0] 1786352736.262269 mapwrite 0x68288c thru 0x683474 (0xbe9 bytes) 000370[0] 1786352736.417242 read 0x14e1e2 thru 0x14eaca (0x8e9 bytes) 000373[1] 1786352736.510780 read 0x113f6f thru 0x115923 (0x19b5 bytes) 000377[0] 1786352736.554210 read 0x127433 thru 0x1287ac (0x137a bytes) 000380[1] 1786352736.639168 mapread 0x463098 thru 0x464d22 (0x1c8b bytes) 000381[1] 1786352736.669994 trunc from 0x6a173f to 0x76f7a8 (0xce06a bytes) 000384[0] 1786352736.797509 mapread 0x515a19 thru 0x515dad (0x395 bytes) 000385[0] 1786352736.857589 write 0x821e8c thru 0x822e27 (0xf9c bytes) 000386[0] 1786352736.907172 mapread 0x50b1ac thru 0x50b44b (0x2a0 bytes) 000388[0] 1786352736.932347 mapwrite 0x6f1522 thru 0x6f1808 (0x2e7 bytes) 000390[1] 1786352736.997995 trunc from 0x822e28 to 0x64f604 (0x1d3823 bytes) 000391[0] 1786352737.079788 mapwrite 0x7e9994 thru 0x7eb517 (0x1b84 bytes) 000392[0] 1786352737.277412 trunc from 0x7eb518 to 0x49abdc (0x35093b bytes) 000395[1] 1786352737.357232 read 0x03cfbd thru 0x03e572 (0x15b6 bytes) 000397[0] 1786352737.402830 mapread 0x47fbf4 thru 0x480e24 (0x1231 bytes) 000400[0] 1786352737.449639 trunc from 0x49abdd to 0x19f7b3 (0x2fb429 bytes) 000401[0] 1786352737.495262 mapwrite 0x6d7fdf thru 0x6d8d16 (0xd38 bytes) 000403[0] 1786352737.606646 mapwrite 0x2bc67c thru 0x2be421 (0x1da6 bytes) 000404[0] 1786352737.675213 mapwrite 0x16329e thru 0x163cc7 (0xa2a bytes) 000405[1] 1786352737.766348 mapread 0x11630d thru 0x11706f (0xd63 bytes) 000409[1] 1786352737.844376 read 0x1eb9c4 thru 0x1ebeb8 (0x4f5 bytes) 000410[1] 1786352737.851922 trunc from 0x6d8d17 to 0x82cd3e (0x154028 bytes) 000411[0] 1786352737.943766 trunc from 0x82cd3f to 0x666963 (0x1c63db bytes) 000413[0] 1786352738.054636 mapwrite 0x791a89 thru 0x7928fb (0xe73 bytes) 000416[0] 1786352738.132284 read 0x38e94d thru 0x38fc62 (0x1316 bytes) 000417[1] 1786352738.173969 read 0x2bfeae thru 0x2c1358 (0x14ab bytes) 000419[1] 1786352738.216964 trunc from 0x7928fc to 0x59ac0c (0x1f7cef bytes) 000423[1] 1786352738.386258 mapwrite 0x93f691 thru 0x94115d (0x1acd bytes) 000424[1] 1786352738.484493 trunc from 0x94115e to 0x20292f (0x73e82e bytes) 000425[0] 1786352738.507993 write 0x73dd6b thru 0x73dda5 (0x3b bytes) 000426[1] 1786352738.543850 mapread 0x408f6f thru 0x409f92 (0x1024 bytes) 000427[1] 1786352738.584645 mapwrite 0x2de511 thru 0x2e0126 (0x1c16 bytes) 000428[1] 1786352738.679099 read 0x4915b0 thru 0x492229 (0xc7a bytes) 000429[1] 1786352738.687768 write 0x4c9868 thru 0x4cb271 (0x1a0a bytes) 000430[1] 1786352738.717292 write 0x879575 thru 0x87b04e (0x1ada bytes) 000431[1] 1786352738.719170 trunc from 0x87b04f to 0x6c3a02 (0x1b764c bytes) 000432[0] 1786352738.784586 trunc from 0x6c3a03 to 0x0954c9 (0x62e539 bytes) 000436[1] 1786352738.929157 read 0x052c6b thru 0x05351e (0x8b4 bytes) 000441[1] 1786352738.995148 trunc from 0x0954ca to 0x8c8ff1 (0x833b28 bytes) 000448[1] 1786352739.190869 write 0x0515bb thru 0x0521a4 (0xbea bytes) 000449[1] 1786352739.321616 mapread 0x74e37a thru 0x74e547 (0x1ce bytes) 000450[1] 1786352739.396435 trunc from 0x8c8ff2 to 0x8ef2df (0x262ee bytes) 000453[0] 1786352739.561437 mapwrite 0x13a968 thru 0x13b152 (0x7eb bytes) 000456[1] 1786352739.739511 write 0x0c4f14 thru 0x0c5bfa (0xce7 bytes) 000461[0] 1786352739.834237 mapwrite 0x8f33a7 thru 0x8f4552 (0x11ac bytes) 000464[1] 1786352740.008031 read 0x77c981 thru 0x77d6a7 (0xd27 bytes) 000465[0] 1786352740.035175 read 0x862831 thru 0x8646a9 (0x1e79 bytes) 000466[0] 1786352740.041798 read 0x565b9e thru 0x5668fc (0xd5f bytes) 000467[1] 1786352740.070981 write 0x9a6fec thru 0x9a73dc (0x3f1 bytes) 000469[1] 1786352740.102123 trunc from 0x9a73dd to 0x90527c (0xa2160 bytes) 000473[1] 1786352740.193851 trunc from 0x90527d to 0x12b6d0 (0x7d9bac bytes) 000474[0] 1786352740.253054 trunc from 0x12b6d1 to 0x4d2699 (0x3a6fc9 bytes) 000476[0] 1786352740.380230 read 0x172e98 thru 0x173e84 (0xfed bytes) 000477[1] 1786352740.392597 mapread 0x0efb16 thru 0x0f0145 (0x630 bytes) 000478[1] 1786352740.451988 write 0x449cc9 thru 0x44b7e4 (0x1b1c bytes) 000482[0] 1786352740.510119 read 0x433181 thru 0x434b13 (0x1993 bytes) 000483[0] 1786352740.544214 mapwrite 0x104d0d thru 0x104fd6 (0x2ca bytes) 000484[0] 1786352740.656661 write 0x32a2db thru 0x32b77e (0x14a4 bytes) 000485[0] 1786352740.678139 mapwrite 0x804076 thru 0x80481a (0x7a5 bytes) 000490[0] 1786352740.885779 write 0x9f60e5 thru 0x9f66a7 (0x5c3 bytes) 000492[0] 1786352740.942950 write 0x6a3257 thru 0x6a3516 (0x2c0 bytes) 000493[0] 1786352740.992777 write 0x27090e thru 0x2710b5 (0x7a8 bytes) 000494[0] 1786352741.006550 mapwrite 0x6f1347 thru 0x6f14c6 (0x180 bytes) 000496[1] 1786352741.085408 trunc from 0x9f66a8 to 0x9a39b1 (0x52cf6 bytes) 000497[1] 1786352741.175339 trunc from 0x9a39b2 to 0x77b28d (0x228724 bytes) 000500[1] 1786352741.259615 read 0x20732d thru 0x2087ac (0x1480 bytes) All operations completed A-OK! PASS 16b (27s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 05:05:46 (1786352746) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (4s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 05:05:50 (1786352750) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.464982 s, 18.0 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.391198 s, 21.4 MB/s ldlm.namespaces.MGC192.168.204.110@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff98c8c2eba800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff98c8d1e4f800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff98c8c2eba800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff98c8d1e4f800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff98c8c2eba800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff98c8d1e4f800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.17407 s, 11.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.86208 s, 36.6 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.31553 s, 45.3 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.70201 s, 22.3 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.77843 s, 15.5 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 3.60829 s, 29.1 MB/s PASS 16d (41s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 05:06:31 (1786352791) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.265044 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127121 s, 8.2 MB/s PASS 16e (5s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 05:06:36 (1786352796) TIMEOUT PASS 16f (26s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 05:07:03 (1786352823) TIMEOUT PASS 16g (27s) == sanityn test 16h: mmap read after truncate file ======= 05:07:30 (1786352850) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16h (7s) == sanityn test 16i: read after truncate file ============ 05:07:36 (1786352856) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (5s) == sanityn test 16j: race dio with buffered i/o ========== 05:07:42 (1786352862) 1+0 records in 1+0 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.49158 s, 17.6 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.01382 s, 25.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.2731 s, 11.5 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.69175 s, 15.5 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.96638 s, 13.3 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.7448 s, 15.0 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.03175 s, 12.9 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.46892 s, 17.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.57871 s, 16.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.34529 s, 11.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.48704 s, 10.5 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.7043 s, 15.4 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.98343 s, 13.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.51252 s, 10.4 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.62882 s, 10.0 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.99934 s, 13.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.37144 s, 11.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.78381 s, 9.4 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.87426 s, 9.1 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (30s) == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 05:08:12 (1786352892) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 All operations completed A-OK! Mon Aug 10 05:08:48 EDT 2026 + fsx 2 done: rc=0 Mon Aug 10 05:08:48 EDT 2026 + fsx 3 done: rc=0 Mon Aug 10 05:08:48 EDT 2026 All operations completed A-OK! + fsx 4 done: rc=0 Mon Aug 10 05:08:48 EDT 2026 + fsx 5 done: rc=0 Mon Aug 10 05:08:48 EDT 2026 PASS 16k (43s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 05:08:55 (1786352935) fail_loc=0x8000030a PASS 17 (8s) == sanityn test 18: mmap sanity check =========================================================================================== 05:09:03 (1786352943) mmap test1: basic mmap operation (PASS, 0.11248s) mmap test2: MAP_PRIVATE not write back (PASS, 0.062254s) mmap test3: concurrent mmap ops on two nodes (PASS, 20.637s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 4.3807s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.11191s) mmap test6: check mmap write/read content on two nodes (PASS, 0.14533s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: multi-thread mmap access (PASS, 0.69774s) mmap test11: multi-thread shared mmap (PASS, 0.80669s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0548s) PASS 18 (34s) == sanityn test 19: test concurrent uncached read races ========================================================================= 05:09:37 (1786352977) oleg410-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg410-client: oleg410-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (3s) == sanityn test 20: test extra readahead page left in cache ============================================================== 05:09:40 (1786352980) PASS 20 (4s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 05:09:45 (1786352985) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (4s) == sanityn test 23: others should see updated atime while another read============================================================== 05:09:49 (1786352989) now is 1786352990 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 new atime is 1786353051 PASS 23 (65s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 05:10:55 (1786353055) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 2176 2206464 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 2176 2206464 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 524.8K 386 524.4K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.3K 403 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 114.5K 400 114.1K 1% /mnt/lustre[OST:1] filesystem_summary: 229.4K 386 229.0K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 524.8K 386 524.4K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 403 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 400 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.4K 386 229.0K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 2.1M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 32.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 38.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 537416 386 537030 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118067 403 117664 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117232 400 116832 1% /mnt/lustre2[OST:1] filesystem_summary: 234882 386 234496 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 2176 2206464 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 524.8K 386 524.4K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 403 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 400 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.4K 386 229.0K 1% /mnt/lustre2 Reader error: 'No devices found' at 0 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 537416 386 537030 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118067 403 117664 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 117232 400 116832 1% /mnt/lustre[OST:1] filesystem_summary: 234882 386 234496 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 537416 386 537030 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118067 403 117664 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117232 400 116832 1% /mnt/lustre2[OST:1] filesystem_summary: 234882 386 234496 1% /mnt/lustre2 Reader error: 'No devices found' at 0 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 2176 2206464 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 2176 2206464 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 32768 3736576 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre2 PASS 24a (5s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 05:10:59 (1786353059) PASS 24b (3s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 05:11:02 (1786353062) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (5s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 05:11:07 (1786353067) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (2s) == sanityn test 26a: allow mtime to get older ============ 05:11:09 (1786353069) utime: good mknod times 1786353069 <= 1786353070 <= 1786353070 for /mnt/lustre/f26a utime: good mknod times 1786353069 <= 1786353070 <= 1786353070 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (5s) == sanityn test 26b: sync mtime between ost and mds ====== 05:11:14 (1786353074) PASS 26b (6s) == sanityn test 26c: set-in-past on open file is not lost on close ========================================================== 05:11:20 (1786353080) multiop /mnt/lustre/f26c.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 File: /mnt/lustre/f26c.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502396 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2026-08-10 05:11:21.000000000 -0400 Birth: 2026-08-10 05:11:21.000000000 -0400 File: /mnt/lustre2/f26c.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502396 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2026-08-10 05:11:21.000000000 -0400 Birth: 2026-08-10 05:11:21.000000000 -0400 PASS 26c (6s) SKIP: sanityn test_27 skipping excluded test 27 == sanityn test 30: recreate file race =================== 05:11:27 (1786353087) 0 PASS 30 (7s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 05:11:34 (1786353094) fail_loc=0x314 PASS 31a (6s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 05:11:40 (1786353100) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg410-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.110@tcp:/lustre 7542784 39936 7498752 1% /mnt/lustre2 PASS 31b (18s) == sanityn test 31r: open-rename(replace) race =========== 05:11:58 (1786353118) -rw-r--r-- 1 root root 0 Aug 10 05:11 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (8s) == sanityn test 31s: open should not revalidate invalid dentry ========================================================== 05:12:06 (1786353126) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 1205: 43130 Terminated tail -f $tfile PASS 31s (5s) == sanityn test 31t: getattr should not revalidate invalid dentry ========================================================== 05:12:11 (1786353131) PASS 31t (6s) == sanityn test 32b: lockless i/o ======================== 05:12:17 (1786353137) SKIP: sanityn test_32b max_nolock_bytes is removed >= 2.17.53 SKIP 32b (2s) == sanityn test 32c: contention detection ================ 05:12:19 (1786353139) ldlm.namespaces.filter-lustre-OST0000_UUID.contended_locks=2 ldlm.namespaces.filter-lustre-OST0001_UUID.contended_locks=2 ldlm.namespaces.filter-lustre-OST0000_UUID.contention_events=0 ldlm.namespaces.filter-lustre-OST0001_UUID.contention_events=0 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.795065 s, 65.9 MB/s 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.676542 s, 77.5 MB/s 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.03886 s, 50.5 MB/s 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.25824 s, 41.7 MB/s contention_seconds 4, contention_events 18 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.28311 s, 40.9 MB/s 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.45585 s, 36.0 MB/s ldlm.namespaces.filter-lustre-OST0000_UUID.contention_events=0 ldlm.namespaces.filter-lustre-OST0001_UUID.contention_events=0 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.795613 s, 65.9 MB/s 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.890539 s, 58.9 MB/s PASS 32c (17s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 05:12:37 (1786353157) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (3s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 05:12:40 (1786353160) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (2s) == sanityn test 33d: dependent transactions should trigger COS ========================================================== 05:12:42 (1786353162) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (2s) == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 05:12:45 (1786353165) SKIP: sanityn test_33e needs >= 2 MDTs SKIP 33e (2s) == sanityn test 34: no lock timeout under IO ============= 05:12:47 (1786353167) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg410-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff98c8c2eba800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff98c8c2eba800.ost_server_uuid in FULL state after 0 sec oleg410-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff98c8c2eba800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff98c8c2eba800.ost_server_uuid in IDLE state after 0 sec oleg410-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff98c8c2eba800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff98c8c2eba800.ost_server_uuid in FULL state after 0 sec oleg410-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff98c8c2eba800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff98c8c2eba800.ost_server_uuid in IDLE state after 0 sec oleg410-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff98c8c2eba800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff98c8c2eba800.ost_server_uuid in FULL state after 0 sec oleg410-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff98c8c2eba800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff98c8c2eba800.ost_server_uuid in IDLE state after 0 sec PASS 34 (111s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 05:14:38 (1786353278) fail_loc=0x80000317 Race attempt 0 Wait for 50425 50519 for 60 sec... - open/close 1093 (time 1786353292.05 total 10.00 last 109.27) - open/close 2223 (time 1786353302.06 total 20.00 last 112.99) - open/close 3368 (time 1786353312.06 total 30.01 last 114.41) total: 4000 open/close in 36.01 seconds: 111.07 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.204.110@tcp:/lustre 7.2G 39M 7.2G 1% /mnt/lustre 192.168.204.110@tcp:/lustre 7.2G 39M 7.2G 1% /mnt/lustre2 PASS 35 (67s) == sanityn test 36: handle ESTALE/open-unlink correctly == 05:15:46 (1786353346) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg410-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.850771 s, 61.6 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7498752) after_dd(7441408) after(7492608) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.691566 s, 75.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7492608) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.706572 s, 74.2 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7494656) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.799673 s, 65.6 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7496704) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.741567 s, 70.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7494656) after_dd(7441408) after(7492608) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.794723 s, 66.0 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7492608) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.629708 s, 83.3 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7494656) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.674345 s, 77.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7494656) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.676062 s, 77.6 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7494656) after_dd(7441408) after(7492608) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.730598 s, 71.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7492608) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.717407 s, 73.1 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7494656) after_dd(7443456) after(7494656) PASS 36 (276s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 05:20:21 (1786353621) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 - create 2441 (time 1786353633.68 total 10.00 last 244.10) - create 4491 (time 1786353643.69 total 20.00 last 204.93) - create 6680 (time 1786353653.69 total 30.01 last 218.86) - create 8805 (time 1786353663.69 total 40.01 last 212.50) total: 10000 create in 46.75 seconds: 213.90 ops/second PASS 37 (53s) == sanityn test 39a: file mtime does not change after rename ========================================================== 05:21:14 (1786353674) repeat after cancel_lru_locks PASS 39a (4s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 05:21:18 (1786353678) repeat after cancel_lru_locks PASS 39b (5s) == sanityn test 39c: check truncate mtime update ================================================================================ 05:21:23 (1786353683) repeat after cancel_lru_locks PASS 39c (5s) == sanityn test 39d: sync write should update mtime ====== 05:21:28 (1786353688) fail_loc=0x411 fail_loc=0 PASS 39d (4s) SKIP: sanityn test_40a skipping ALWAYS excluded test 40a == sanityn test 40b: pdirops: open|create and others ======================================================================== 05:21:33 (1786353693) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (11s) == sanityn test 40c: pdirops: link and others ======================================================================== 05:21:43 (1786353703) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (11s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 05:21:54 (1786353714) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (10s) == sanityn test 40e: pdirops: rename and others ======================================================================== 05:22:04 (1786353724) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (10s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 05:22:14 (1786353734) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (7s) == sanityn test 41b: pdirops: create vs create ======================================================================== 05:22:21 (1786353741) open: File exists Conflict PASS 41b (8s) == sanityn test 41c: pdirops: create vs link ======================================================================== 05:22:29 (1786353749) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (8s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 05:22:37 (1786353757) Conflict PASS 41d (7s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 05:22:44 (1786353764) Conflict PASS 41e (8s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 05:22:52 (1786353772) Conflict PASS 41f (7s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 05:22:59 (1786353779) Conflict PASS 41g (7s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 05:23:06 (1786353786) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 2427: lctl set_param -n fail_loc=0 2>/dev/null || true: No such file or directory Conflict PASS 41h (8s) == sanityn test 41i: reint_open: create vs create ======== 05:23:14 (1786353794) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (760s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 05:35:54 (1786354554) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (5s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 05:35:59 (1786354559) open: File exists Conflict PASS 42b (5s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 05:36:04 (1786354564) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (5s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 05:36:09 (1786354569) Conflict PASS 42d (5s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 05:36:14 (1786354574) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (5s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 05:36:19 (1786354579) Conflict PASS 42f (4s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 05:36:24 (1786354584) Conflict PASS 42g (4s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 05:36:28 (1786354588) Conflict PASS 42h (5s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 05:36:33 (1786354593) PASS 43a (17s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 05:36:51 (1786354611) Conflict PASS 43b (5s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 05:36:56 (1786354616) Conflict PASS 43c (5s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 05:37:01 (1786354621) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (5s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 05:37:06 (1786354626) Conflict PASS 43e (5s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 05:37:11 (1786354631) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (5s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 05:37:16 (1786354636) stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (5s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 05:37:21 (1786354641) Conflict PASS 43h (5s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 05:37:26 (1786354646) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 05:37:27 (1786354647) mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (36s) == sanityn test 43k: unlink vs create ==================== 05:38:03 (1786354683) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (433s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 05:45:16 (1786355116) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Mon Aug 10 05:45:19 EDT 2026 PASS 44a (5s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 05:45:21 (1786355121) open: File exists Conflict PASS 44b (5s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 05:45:26 (1786355126) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (6s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 05:45:32 (1786355132) Conflict PASS 44d (5s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 05:45:37 (1786355137) Conflict PASS 44e (5s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 05:45:42 (1786355142) Conflict PASS 44f (5s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 05:45:47 (1786355147) Conflict PASS 44g (5s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 05:45:52 (1786355152) Conflict PASS 44h (5s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 05:45:57 (1786355157) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 05:45:58 (1786355158) PASS 45a (31s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 05:46:29 (1786355189) Conflict PASS 45b (6s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 05:46:35 (1786355195) Conflict PASS 45c (5s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 05:46:40 (1786355200) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (5s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 05:46:45 (1786355205) Conflict PASS 45e (6s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 05:46:51 (1786355211) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (5s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 05:46:56 (1786355216) stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (5s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 05:47:01 (1786355221) Conflict PASS 45h (5s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 05:47:06 (1786355226) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 05:47:07 (1786355227) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (432s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 05:54:19 (1786355659) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (5s) == sanityn test 46b: pdirops: link vs create ======================================================================== 05:54:25 (1786355665) open: File exists Conflict PASS 46b (5s) == sanityn test 46c: pdirops: link vs link ======================================================================== 05:54:30 (1786355670) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (5s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 05:54:35 (1786355675) Conflict PASS 46d (6s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 05:54:41 (1786355681) Conflict PASS 46e (5s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 05:54:46 (1786355686) Conflict PASS 46f (5s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 05:54:52 (1786355692) Conflict PASS 46g (6s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 05:54:57 (1786355697) Conflict PASS 46h (5s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 05:55:02 (1786355702) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 05:55:03 (1786355703) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 05:55:04 (1786355704) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 05:55:05 (1786355705) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (2s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 05:55:07 (1786355707) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 05:55:08 (1786355708) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 05:55:09 (1786355709) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 05:55:10 (1786355710) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 05:55:11 (1786355711) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.00581323 s, 1.8 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 05:55:18 (1786355718) 0+1 records in 0+1 records out 158 bytes copied, 0.00446405 s, 35.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3690: kill: (218889) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 05:55:22 (1786355722) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00328427 s, 312 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210935 s, 49.7 MB/s 3145728 PASS 51b (15s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 05:55:37 (1786355737) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00431826 s, 237 kB/s PASS 51c (7s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 05:55:44 (1786355744) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132162 s, 79.3 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (3s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 05:55:47 (1786355747) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2454 0x996 0x240000400 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2454 0x996 0x240000400 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 05:55:51 (1786355751) llite.lustre-ffff98c8c2eba800.xattr_cache=1 llite.lustre-ffff98c8d1e4f800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505012 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-08-10 05:55:52.000000000 -0400 Modify: 2026-08-10 05:55:52.000000000 -0400 Change: 2026-08-10 05:55:52.000000000 -0400 Birth: 2026-08-10 05:55:51.000000000 -0400 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 7680 Blocks: 15 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505015 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-08-10 05:55:57.000000000 -0400 Modify: 2026-08-10 05:55:58.000000000 -0400 Change: 2026-08-10 05:55:58.000000000 -0400 Birth: 2026-08-10 05:55:57.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 7680 Blocks: 15 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505018 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-08-10 05:56:03.000000000 -0400 Modify: 2026-08-10 05:56:03.000000000 -0400 Change: 2026-08-10 05:56:03.000000000 -0400 Birth: 2026-08-10 05:56:02.000000000 -0400 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 7680 Blocks: 15 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505017 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-08-10 05:56:03.000000000 -0400 Modify: 2026-08-10 05:56:03.000000000 -0400 Change: 2026-08-10 05:56:03.000000000 -0400 Birth: 2026-08-10 05:56:02.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1/d2 Size: 7680 Blocks: 15 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505021 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-08-10 05:56:08.000000000 -0400 Modify: 2026-08-10 05:56:08.000000000 -0400 Change: 2026-08-10 05:56:08.000000000 -0400 Birth: 2026-08-10 05:56:08.000000000 -0400 File: /mnt/lustre/d1 Size: 7680 Blocks: 15 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505020 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-08-10 05:56:08.000000000 -0400 Modify: 2026-08-10 05:56:08.000000000 -0400 Change: 2026-08-10 05:56:08.000000000 -0400 Birth: 2026-08-10 05:56:08.000000000 -0400 PASS 54 (24s) == sanityn test 55a: rename vs unlink target dir ========= 05:56:15 (1786355775) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (8s) == sanityn test 55b: rename vs unlink source dir ========= 05:56:23 (1786355783) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 05:56:30 (1786355790) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (13s) == sanityn test 55d: rename file vs link ================= 05:56:43 (1786355803) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (10s) == sanityn test 55e: rename race AB/BA under the same parent dir ========================================================== 05:56:53 (1786355813) SKIP: sanityn test_55e needs >= 2 MDTs SKIP 55e (1s) == sanityn test 55f: rename: (P1/A -> P2/B) race with (P2/B -> P1/A) ========================================================== 05:56:54 (1786355814) fail_loc=0x155 We survived after P1A_P2B/P2B_P1A race test fail_loc=0x0 PASS 55f (13s) == sanityn test 55g: rename: race with trylock =========== 05:57:07 (1786355827) mdt.lustre-MDT0000.enable_rename_trylock=1 fail_loc=0x155 mv: cannot move '/mnt/lustre/d55g.sanityn/a1/a2' to a subdirectory of itself, '/mnt/lustre/d55g.sanityn/b1/b2/b3/b4/a2' We survived race with trylock test fail_loc=0x0 PASS 55g (14s) == sanityn test 56a: test llverdev with single large stripe ========================================================== 05:57:21 (1786355841) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1786355842 chunksize: 4096 size: 16777216000000 write offset: 16384000000kB 2.54329 MB/s read offset: 16384000000kB inf MB/s llverdev: data verified successfully PASS 56a (27s) == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 05:57:48 (1786355868) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1786355870 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 3.48783 MB/s read offset: 32768000000kB inf MB/s llverdev: data verified successfully PASS 56b (39s) == sanityn test 60: Verify data_version behaviour ======== 05:58:28 (1786355908) PASS 60 (2s) == sanityn test 70a: cd directory && rm directory ======== 05:58:30 (1786355910) PASS 70a (2s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 05:58:32 (1786355912) PASS 70b (3s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 05:58:35 (1786355915) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00300968 s, 21.8 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 05:58:36 (1786355916) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00504384 s, 13.0 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (2s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 05:58:38 (1786355918) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 05:58:39 (1786355919) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00401145 s, 16.3 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 05:58:40 (1786355920) llite.lustre-ffff98c8c2eba800.xattr_cache=1 llite.lustre-ffff98c8d1e4f800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 05:58:42 (1786355922) llite.lustre-ffff98c8c2eba800.xattr_cache=1 llite.lustre-ffff98c8d1e4f800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (3s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 05:58:45 (1786355925) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.003189 s, 321 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00258761 s, 396 kB/s 231645: get lock2 231646: get lock1 231645: done 231645: get lock1 231646: done 231646: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 231645: cannot get lock1: Resource deadlock avoided 231646: done 231646: exit rc=0 231645: done 231645: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 05:58:51 (1786355931) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0281428 s, 74.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109595 s, 95.7 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4373: 232227 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 05:58:58 (1786355938) opening files: ........ closing files: ........ PASS 76 (21s) == sanityn test 77a: check FIFO NRS policy =============== 05:59:19 (1786355959) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.137752 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0493673 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0460684 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0847889 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127654 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12924 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131031 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138472 s, 7.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff98c8d1e4f800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff98c8d1e4f800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183249 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182518 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197914 s, 5.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187201 s, 5.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192823 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203677 s, 5.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218587 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205584 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230545 s, 4.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff98c8d1e4f800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff98c8d1e4f800.lock_unused_count=5 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216639 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193306 s, 54.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232416 s, 45.1 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176814 s, 59.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736985 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173904 s, 60.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168648 s, 62.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218913 s, 47.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153342 s, 68.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271739 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031581 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309484 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330142 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132619 s, 79.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010823 s, 96.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211584 s, 49.6 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (3s) == sanityn test 77b: check CRR-N NRS policy ============== 05:59:22 (1786355962) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.166146 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0920278 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1234 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132427 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137741 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142783 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127098 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149629 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162747 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138932 s, 7.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180109 s, 5.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159237 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172889 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166052 s, 6.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff98c8d1e4f800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149228 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179727 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148816 s, 7.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015166 s, 69.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254333 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190822 s, 55.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190823 s, 55.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285498 s, 36.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257708 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0340412 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143804 s, 72.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309518 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00604605 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102643 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322257 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193712 s, 54.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796498 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152461 s, 68.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00174087 s, 602 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg410-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg410-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.139354 s, 120 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0489958 s, 21.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0614649 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102495 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100006 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137222 s, 7.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137435 s, 7.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141026 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114276 s, 9.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff98c8d1e4f800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161798 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146911 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148247 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144894 s, 7.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138455 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116715 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123802 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124208 s, 8.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303961 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018733 s, 56.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130811 s, 80.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171712 s, 61.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187375 s, 56.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.037542 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273322 s, 38.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.034238 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155414 s, 67.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0437184 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134667 s, 77.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0411817 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014164 s, 74.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146712 s, 71.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023022 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106133 s, 98.8 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (4s) == sanityn test 77c: check ORR NRS policy ================ 05:59:26 (1786355966) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.177453 s, 94.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262726 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0406264 s, 25.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152927 s, 6.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163994 s, 6.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172355 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165846 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172108 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158752 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186734 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182093 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165895 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171956 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164789 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180225 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189179 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205613 s, 5.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff98c8d1e4f800.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff98c8d1e4f800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187168 s, 56.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198291 s, 52.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181507 s, 57.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247221 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270382 s, 38.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235597 s, 44.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184033 s, 57.0 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0359491 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138042 s, 76.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0340616 s, 30.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121445 s, 86.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106353 s, 98.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802439 s, 131 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276445 s, 37.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0325085 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00458973 s, 228 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.136446 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0499886 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0830737 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100002 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0980821 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111842 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135341 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119479 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144122 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114793 s, 9.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134615 s, 7.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154695 s, 6.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127371 s, 8.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142559 s, 7.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158147 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175004 s, 6.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166929 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170396 s, 61.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330883 s, 31.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269112 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00536877 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226447 s, 46.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153894 s, 68.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0419565 s, 25.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0420777 s, 24.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff98c8d1e4f800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff98c8d1e4f800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252164 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280506 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0453396 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0499997 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0377265 s, 27.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0387412 s, 27.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00426416 s, 246 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0403286 s, 26.0 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.131612 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.040204 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0396346 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0628911 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102786 s, 10.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119536 s, 8.8 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0973135 s, 10.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101151 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0925414 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10721 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122066 s, 8.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff98c8d1e4f800.lock_unused_count=2 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111034 s, 9.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110014 s, 9.5 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112385 s, 9.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107812 s, 9.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127773 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11601 s, 9.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019621 s, 53.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190224 s, 55.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244256 s, 42.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023253 s, 45.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275917 s, 38.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02376 s, 44.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00980145 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243711 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643822 s, 163 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201837 s, 52.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124437 s, 84.3 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff98c8d1e4f800.lock_unused_count=1 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026446 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120304 s, 87.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013916 s, 75.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131194 s, 79.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197199 s, 53.2 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (5s) == sanityn test 77d: check TRR nrs policy ================ 05:59:31 (1786355971) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.214688 s, 78.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280812 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435751 s, 24.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139999 s, 7.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137033 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118371 s, 8.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff98c8d1e4f800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff98c8d1e4f800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144431 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122187 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152291 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144396 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164875 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160169 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134105 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158094 s, 6.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175109 s, 6.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168012 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148804 s, 7.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01519 s, 69.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233238 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198598 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181517 s, 57.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274336 s, 38.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336175 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204368 s, 51.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148628 s, 70.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013863 s, 75.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309954 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296425 s, 35.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190886 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137416 s, 76.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292706 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00530805 s, 198 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146227 s, 71.7 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.137478 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0414872 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0566892 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0518285 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0843611 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0742784 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0724212 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0898028 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104626 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0968636 s, 10.8 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff98c8d1e4f800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff98c8d1e4f800.lock_unused_count=2 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241653 s, 43.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129061 s, 81.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155283 s, 6.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323726 s, 32.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171676 s, 61.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163766 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.035023 s, 29.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136384 s, 76.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185755 s, 5.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168535 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240735 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201861 s, 51.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0444123 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236356 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320844 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181775 s, 5.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0388697 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170924 s, 61.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267171 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220938 s, 47.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201634 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0385518 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187868 s, 5.6 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.138432 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0547867 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0568585 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0778268 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0507554 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0913036 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0685934 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0735304 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103636 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0844006 s, 12.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019848 s, 52.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023375 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0346561 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163899 s, 64.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186992 s, 5.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18362 s, 5.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243971 s, 43.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181399 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17884 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183189 s, 5.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190383 s, 5.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189876 s, 55.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0352737 s, 29.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791106 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179776 s, 58.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199375 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0473715 s, 22.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0354705 s, 29.6 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0001-osc-ffff98c8d1e4f800.lock_unused_count=8 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194846 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00485209 s, 216 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0348015 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0401392 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020112 s, 52.1 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg410-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg410-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (5s) == sanityn test 77e: check TBF NID nrs policy ============ 05:59:36 (1786355976) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.204.10@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.195126 s, 86.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230695 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028578 s, 36.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0754733 s, 13.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0855669 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0998621 s, 10.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0809369 s, 13.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0975729 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101494 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0944934 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860204 s, 12.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0971563 s, 10.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0932063 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0921247 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0894129 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0922832 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.088027 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240249 s, 43.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179631 s, 58.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022255 s, 47.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226076 s, 46.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264161 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029664 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283412 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180255 s, 58.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835866 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00544465 s, 193 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824303 s, 127 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0421696 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269439 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164833 s, 63.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173967 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0355881 s, 29.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg410-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg410-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.133735 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372889 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0640204 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0910006 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101038 s, 10.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107573 s, 9.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113848 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102455 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125986 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103293 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153042 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122927 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149245 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144065 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147148 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136628 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15271 s, 6.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff98c8d1e4f800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff98c8d1e4f800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142036 s, 7.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144691 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15623 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142335 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142431 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14753 s, 7.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149367 s, 7.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147553 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160647 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150967 s, 6.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151689 s, 6.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149936 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148087 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153519 s, 6.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150553 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167195 s, 6.3 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.180317 s, 93.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0602153 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0423732 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0867145 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0965571 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0790298 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0976468 s, 10.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0937887 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0948988 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148752 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15281 s, 6.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171473 s, 61.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099318 s, 106 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240016 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171056 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172908 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19247 s, 5.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341312 s, 30.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18517 s, 5.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975784 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0541327 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168272 s, 6.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284152 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247587 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915341 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206643 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248578 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211505 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0641309 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258716 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118197 s, 88.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.036606 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0653975 s, 16.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff98c8d1e4f800.lock_unused_count=1 removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg410-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg410-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.138159 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200064 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0503676 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0763779 s, 13.7 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff98c8d1e4f800.lock_unused_count=2 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0885416 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0986321 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10606 s, 9.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0721334 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.088345 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0873368 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121093 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126098 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0965621 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107966 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823961 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105447 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119805 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156282 s, 67.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0345285 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260592 s, 40.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0095233 s, 110 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff98c8d1e4f800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff98c8d1e4f800.lock_unused_count=2 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183966 s, 57.0 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0376003 s, 27.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320452 s, 32.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309583 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0387732 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0368263 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178873 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217186 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0357431 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291496 s, 36.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284042 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280923 s, 37.3 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (9s) == sanityn test 77f: check TBF JobID nrs policy ========== 05:59:45 (1786355985) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.205066 s, 81.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0388143 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0350039 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0811885 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129217 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129562 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14353 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130641 s, 8.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff98c8d1e4f800.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff98c8d1e4f800.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181487 s, 5.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141526 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173935 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171448 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175988 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164357 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188771 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176892 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178878 s, 5.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff98c8d1e4f800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff98c8d1e4f800.lock_unused_count=2 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014808 s, 70.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154042 s, 68.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209606 s, 50.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191535 s, 54.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021304 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244227 s, 42.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136996 s, 76.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013132 s, 79.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255045 s, 41.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161252 s, 65.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784877 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00995341 s, 105 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134647 s, 77.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033408 s, 31.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213344 s, 49.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149361 s, 70.2 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg410-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg410-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.127759 s, 131 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.038154 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0583362 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0931257 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0845578 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0911839 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103821 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.098408 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.087664 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0831549 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106698 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0979377 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120118 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0938226 s, 11.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10114 s, 10.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105507 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0985506 s, 10.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912324 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160832 s, 65.2 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150779 s, 69.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132502 s, 79.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101063 s, 104 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020754 s, 50.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224356 s, 46.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211373 s, 49.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654616 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238546 s, 44.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246763 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.005352 s, 196 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00450123 s, 233 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992029 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00386164 s, 272 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00474615 s, 221 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.130214 s, 129 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.035651 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0821693 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109721 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116083 s, 9.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144499 s, 7.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129701 s, 8.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173713 s, 6.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146453 s, 7.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153468 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14668 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163301 s, 6.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175718 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168299 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195001 s, 5.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193978 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175876 s, 6.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied ldlm.namespaces.lustre-OST0000-osc-ffff98c8d1e4f800.lock_unused_count=2 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021106 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206595 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017565 s, 59.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195079 s, 53.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231736 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00433175 s, 242 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257176 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196182 s, 53.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654241 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010743 s, 97.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269891 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281886 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230283 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993861 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174512 s, 60.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160596 s, 65.3 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.135294 s, 124 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381798 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0459759 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103096 s, 10.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120952 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121358 s, 8.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116564 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133126 s, 7.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff98c8d1e4f800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff98c8d1e4f800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120347 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125812 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126086 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149564 s, 7.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14338 s, 7.3 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163037 s, 6.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158419 s, 6.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148409 s, 7.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18007 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200972 s, 52.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211576 s, 49.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198349 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236503 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856111 s, 122 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00489866 s, 214 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00897318 s, 117 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016248 s, 64.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272272 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00461602 s, 227 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014298 s, 73.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187088 s, 56.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116023 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00369319 s, 284 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207512 s, 50.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00582147 s, 180 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg410-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg410-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (7s) == sanityn test 77g: Change TBF type directly ============ 05:59:52 (1786355992) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (4s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 05:59:56 (1786355996) oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg410-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg410-client: oleg410-server: ssh exited with exit code 2 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Invalid argument oleg410-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Invalid argument oleg410-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Invalid argument oleg410-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 PASS 77h (4s) == sanityn test 77i: Change rank of TBF rule ============= 06:00:00 (1786356000) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (8s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 06:00:08 (1786356008) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.144687 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299204 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0523822 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0596114 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0701694 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0625242 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0585285 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0664766 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0662648 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0567843 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.085211 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.079761 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107234 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105221 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106721 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114649 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10323 s, 10.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169774 s, 61.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199599 s, 52.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202703 s, 51.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151851 s, 69.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302249 s, 34.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307513 s, 34.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0343883 s, 30.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121413 s, 86.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287968 s, 36.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157616 s, 66.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791997 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206787 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110901 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165311 s, 63.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267605 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155317 s, 67.5 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.94156 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4321 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (48s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 06:00:56 (1786356056) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg410-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg410-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg410-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg410-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.457822 s, 36.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0538126 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0566077 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0649673 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0903002 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0894377 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0729628 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0912547 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.091635 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105877 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107168 s, 9.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11427 s, 9.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136128 s, 7.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131636 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111067 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129185 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127353 s, 8.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159786 s, 65.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018773 s, 55.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198163 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199612 s, 52.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157071 s, 66.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180727 s, 58.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183783 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263132 s, 39.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00398771 s, 263 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100196 s, 105 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021866 s, 48.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00439164 s, 239 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111177 s, 94.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0041251 s, 254 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595666 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00399444 s, 263 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8355 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9556 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg410-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg410-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg410-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg410-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.495341 s, 33.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0707617 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0886485 s, 11.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0794822 s, 13.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0975866 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.096703 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101819 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0825911 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0771118 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111018 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100918 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0945226 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0903874 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1025 s, 10.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0969624 s, 10.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0805236 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.092315 s, 11.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259457 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296749 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586584 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293691 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024704 s, 42.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017243 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0374172 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279649 s, 37.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106733 s, 98.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0394391 s, 26.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190422 s, 55.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198343 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663151 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0397568 s, 26.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0410579 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00522948 s, 201 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg410-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg410-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.7065 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9134 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (134s) == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 06:03:10 (1786356190) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg410-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg410-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg410-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg410-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.23524 s, 13.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324431 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0564298 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0569094 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0484614 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.08175 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0728958 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0685887 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0649403 s, 16.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122336 s, 8.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122257 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119268 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13035 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137641 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131524 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162396 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172868 s, 6.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183089 s, 57.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218204 s, 48.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202906 s, 51.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252121 s, 41.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155026 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212568 s, 49.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148917 s, 70.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242446 s, 43.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104609 s, 100 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028813 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102101 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149071 s, 70.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240091 s, 43.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746295 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025488 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00223602 s, 469 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg410-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg410-client.virtnet' removed directory '/mnt/lustre/d77jb.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8299 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9579 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg410-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg410-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg410-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg410-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.22509 s, 13.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0640327 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0571743 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0671383 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0871888 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0837778 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102097 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0877902 s, 11.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0806217 s, 13.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161271 s, 6.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161308 s, 6.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157156 s, 6.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174327 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171521 s, 6.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167108 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190429 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176249 s, 5.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217874 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746563 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190426 s, 55.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187075 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120105 s, 87.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018008 s, 58.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248998 s, 42.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260568 s, 40.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272869 s, 38.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027933 s, 37.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00402958 s, 260 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026183 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276707 s, 37.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00543406 s, 193 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00429817 s, 244 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00455212 s, 230 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77jb.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8317 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9558 s, 5.3 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (137s) == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 06:05:27 (1786356327) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.360451 s, 46.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0394692 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0474049 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0866868 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0669966 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.081985 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0968892 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107342 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116125 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0930242 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131538 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11608 s, 9.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112715 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111725 s, 9.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115685 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115364 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110554 s, 9.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170438 s, 61.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215933 s, 48.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191706 s, 54.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173625 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145533 s, 72.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189621 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210618 s, 49.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00454834 s, 231 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279495 s, 37.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02961 s, 35.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136564 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00399787 s, 262 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00342024 s, 307 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210144 s, 49.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00326098 s, 322 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289702 s, 36.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.9472 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73489 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.10@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.10@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.203793 s, 82.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435331 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0802636 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0928501 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13009 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123656 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111321 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117875 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1496 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155852 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159634 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180992 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165079 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161257 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18309 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173043 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177415 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00929235 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126206 s, 83.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101329 s, 103 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276242 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203587 s, 51.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193598 s, 54.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233956 s, 44.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230059 s, 45.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238692 s, 43.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103783 s, 101 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111826 s, 93.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136956 s, 76.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025415 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0144462 s, 72.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870281 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00385252 s, 272 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.95098 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74392 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.10@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.126316 s, 133 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0355298 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0914887 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0877032 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0752706 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0698793 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0952904 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0831784 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105929 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0967057 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136807 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154694 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150372 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153172 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153247 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144593 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167025 s, 6.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188257 s, 55.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023519 s, 44.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022938 s, 45.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185539 s, 56.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234955 s, 44.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187847 s, 55.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222376 s, 47.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284725 s, 36.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126819 s, 82.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00962968 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810098 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886512 s, 118 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993072 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191614 s, 54.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154201 s, 68.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184219 s, 56.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.92974 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.87871 s, 21.5 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.128834 s, 130 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206461 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0525751 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269192 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0716613 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0712925 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0635551 s, 16.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0689619 s, 15.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0739354 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0840973 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129448 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129154 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153709 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.267499 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.259279 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.306753 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.326441 s, 3.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165196 s, 63.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205236 s, 51.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245242 s, 42.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255791 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204377 s, 51.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244276 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272287 s, 38.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276564 s, 37.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309033 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013997 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151615 s, 69.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103475 s, 101 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284549 s, 36.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0337528 s, 31.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183351 s, 57.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00482616 s, 217 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg410-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg410-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.78417 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74093 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.9251 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75226 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.220741 s, 76.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0438384 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0599822 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0631515 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0822108 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119549 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106424 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104225 s, 10.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104648 s, 10.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0901967 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0940175 s, 11.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106 s, 9.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126529 s, 8.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103784 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115465 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0973714 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106341 s, 9.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163171 s, 64.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220471 s, 47.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156192 s, 67.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171548 s, 61.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184616 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222488 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224174 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184335 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00549033 s, 191 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838432 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00930887 s, 113 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267063 s, 39.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151822 s, 69.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731181 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00538254 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136457 s, 76.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.5635 s, 5.4 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4769 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.161151 s, 104 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297691 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033928 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123774 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128146 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133969 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130352 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152195 s, 6.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131964 s, 7.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132986 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151684 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149609 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152268 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163867 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159172 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1449 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156055 s, 6.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793693 s, 132 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136105 s, 77.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131299 s, 79.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140923 s, 74.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179656 s, 58.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00937149 s, 112 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163862 s, 64.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180055 s, 58.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00382241 s, 274 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199307 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160157 s, 65.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00408926 s, 256 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00517354 s, 203 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102468 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00239843 s, 437 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00294536 s, 356 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg410-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg410-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.91932 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.134215 s, 125 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211446 s, 49.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326407 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0663666 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0626985 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860313 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0925666 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0761031 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0868004 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0764696 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0843027 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0709845 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0639862 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119385 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102719 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174384 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.21404 s, 4.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128829 s, 81.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014249 s, 73.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017189 s, 61.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009689 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162388 s, 64.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116462 s, 90.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737396 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170414 s, 61.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151672 s, 69.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115675 s, 90.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021928 s, 47.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00387942 s, 270 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225202 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152252 s, 68.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00468242 s, 224 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00469442 s, 223 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.78307 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73046 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.76345 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.72974 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (359s) == sanityn test 77kb: Check different granular TBF type combination: jobid+opcode ========================================================== 06:11:26 (1786356686) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.249013 s, 67.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0416407 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223553 s, 46.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0410206 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0645174 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.056998 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0666766 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0613054 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0730186 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0854044 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.079114 s, 13.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0808456 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0675177 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0894906 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101809 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0719996 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110324 s, 9.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115182 s, 91.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160757 s, 65.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158719 s, 66.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147746 s, 71.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125171 s, 83.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196006 s, 53.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016076 s, 65.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186706 s, 56.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178894 s, 58.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191056 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184981 s, 56.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198463 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607344 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0031034 s, 338 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00301843 s, 347 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00491291 s, 213 MB/s removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77kb.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93725 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.72941 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kb (36s) == sanityn test 77kc: Check different granular TBF type combination: nid+opcode ========================================================== 06:12:02 (1786356722) ost.OSS.ost_io.nrs_policies=tbf nid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.10@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.10@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.245539 s, 68.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020812 s, 50.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0809728 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0760313 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0841675 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0968417 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0930747 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0756194 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823264 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0942553 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111373 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14623 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164503 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.262963 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.264238 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.287655 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.297006 s, 3.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022455 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200178 s, 52.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235789 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195775 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274307 s, 38.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266387 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208854 s, 50.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237982 s, 44.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029153 s, 36.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300506 s, 34.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265364 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292186 s, 35.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282822 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0028407 s, 369 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00332168 s, 316 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00408853 s, 256 MB/s removed '/mnt/lustre/d77kc.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77kc.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77kc.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93493 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73148 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kc.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kc (37s) == sanityn test 77kd: Check different granular TBF type combination: nid+jobid ========================================================== 06:12:39 (1786356759) ost.OSS.ost_io.nrs_policies=tbf nid+jobid ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.10@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kd.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.151396 s, 111 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195599 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0665706 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0764693 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0773271 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102056 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12098 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113151 s, 9.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110731 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100492 s, 10.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112809 s, 9.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123243 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118451 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132823 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11106 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13276 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130298 s, 8.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126798 s, 82.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138608 s, 75.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119403 s, 87.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177999 s, 58.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173456 s, 60.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171157 s, 61.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170162 s, 61.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193994 s, 54.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00950233 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115793 s, 90.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238652 s, 43.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216655 s, 48.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585938 s, 179 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229991 s, 45.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573567 s, 183 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550859 s, 190 MB/s removed '/mnt/lustre/d77kd.sanityn/nrs_r_oleg410-client.virtnet' removed '/mnt/lustre/d77kd.sanityn/nrs_w_oleg410-client.virtnet' removed directory '/mnt/lustre/d77kd.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kd.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93739 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kd.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.96982 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77kd.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kd (35s) == sanityn test 77ke: Check different granular TBF type combination: jobid+opcode ========================================================== 06:13:14 (1786356794) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ke.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.459773 s, 36.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0325162 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.052862 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14039 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143521 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133341 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156847 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181725 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182098 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173409 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165901 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184647 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197571 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.243592 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.233127 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.385044 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.404243 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141038 s, 74.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012582 s, 83.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224827 s, 46.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136926 s, 76.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203121 s, 51.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022252 s, 47.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194545 s, 53.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256023 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196011 s, 53.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176324 s, 59.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257179 s, 40.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268302 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241151 s, 43.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215326 s, 48.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239366 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231946 s, 45.2 MB/s removed '/mnt/lustre/d77ke.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77ke.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77ke.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ke.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.78576 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ke.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73356 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77ke.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93044 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73326 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77ke.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77ke (77s) == sanityn test 77kf: Check different granular TBF type combination: uid+gid ========================================================== 06:14:32 (1786356872) ost.OSS.ost_io.nrs_policies=tbf uid+gid ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kf.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.856344 s, 19.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0331392 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0862703 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127168 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124469 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11102 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124226 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152959 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129382 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163203 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166811 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186969 s, 5.6 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18706 s, 5.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165804 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151862 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183806 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169062 s, 6.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169548 s, 61.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208835 s, 50.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228151 s, 46.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208335 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291333 s, 36.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192242 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101444 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263099 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335367 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254264 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254347 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221938 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00615086 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308047 s, 34.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564743 s, 186 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225702 s, 46.5 MB/s removed '/mnt/lustre/d77kf.sanityn/nrs_r_oleg410-client.virtnet' removed '/mnt/lustre/d77kf.sanityn/nrs_w_oleg410-client.virtnet' removed directory '/mnt/lustre/d77kf.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kf.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6662 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kf.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9529 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77kf.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kf (69s) == sanityn test 77kg: check different granular TBF type combination: uid+opcode ========================================================== 06:15:40 (1786356940) ost.OSS.ost_io.nrs_policies=tbf uid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.334334 s, 50.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0613082 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0960325 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183986 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163236 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179422 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229747 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211368 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18521 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195156 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199337 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19925 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.274864 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240717 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.252262 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.236597 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255405 s, 4.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0440872 s, 23.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023843 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.061605 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0446263 s, 23.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0549329 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0473596 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0477299 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0597328 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0546428 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0806796 s, 13.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0508076 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0556509 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0471988 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0695967 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0816103 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0588286 s, 17.8 MB/s removed '/mnt/lustre/d77kg.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77kg.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77kg.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.36014 s, 19.6 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.79536 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/nrs_r_oleg410-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.699373 s, 24.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115732 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197056 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16207 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229932 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.288201 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.278488 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187836 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229697 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227723 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.444136 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.293684 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.474302 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.376146 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.417085 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.393984 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.171774 s, 6.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277833 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0522015 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0834983 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0632759 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0616147 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0362359 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122432 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0440179 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0520184 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0440806 s, 23.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0707539 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0596702 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0609696 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0692612 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.066553 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.088611 s, 11.8 MB/s removed '/mnt/lustre/d77kg.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77kg.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77kg.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.4105 s, 10.1 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.85819 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.90239 s, 10.6 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.79176 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kg (130s) == sanityn test 77kh: Verify that Project ID support for NRS TBF rule ========================================================== 06:17:50 (1786357070) using SAVE_PROJECT_SUPPORTED=0 Stopping clients: oleg410-client.virtnet /mnt/lustre (opts:) Stopping client oleg410-client.virtnet /mnt/lustre opts: Stopping clients: oleg410-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg410-client.virtnet /mnt/lustre2 opts: Stopping /mnt/lustre-mds1 (opts:-f) on oleg410-server Stopping /mnt/lustre-ost1 (opts:-f) on oleg410-server Stopping /mnt/lustre-ost2 (opts:-f) on oleg410-server sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime) proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) devtmpfs on /dev type devtmpfs (rw,nosuid,mode=755) securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev) devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000) tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755) tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755) cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd) pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime) bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700) cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids) cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio) cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb) cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices) cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,rdma) cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event) cgroup on /sys/fs/cgroup/misc type cgroup (rw,nosuid,nodev,noexec,relatime,misc) cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer) cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct) cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) none on /sys/kernel/tracing type tracefs (rw,relatime) configfs on /sys/kernel/config type configfs (rw,relatime) /dev/nbd0 on / type ext4 (ro,relatime,stripe=32) rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M) mqueue on /dev/mqueue type mqueue (rw,relatime) debugfs on /sys/kernel/debug type debugfs (rw,relatime) none on /mnt type ramfs (rw,relatime) none on /var/lib/stateless/writable type tmpfs (rw,relatime) /dev/vda on /home/green/git/lustre-release type squashfs (ro,relatime) none on /var/cache/man type tmpfs (rw,relatime) none on /var/log type tmpfs (rw,relatime) none on /var/lib/dbus type tmpfs (rw,relatime) none on /tmp type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) none on /var/tmp type tmpfs (rw,relatime) none on /var/lib/NetworkManager type tmpfs (rw,relatime) none on /var/lib/systemd/random-seed type tmpfs (rw,relatime) none on /var/spool type tmpfs (rw,relatime) none on /var/lib/nfs type tmpfs (rw,relatime) none on /var/lib/gssproxy type tmpfs (rw,relatime) none on /var/lib/logrotate type tmpfs (rw,relatime) none on /etc type tmpfs (rw,relatime) none on /var/lib/rsyslog type tmpfs (rw,relatime) none on /root/.ssh type tmpfs (rw,relatime) none on /var/lib/dhclient type tmpfs (rw,relatime) 192.168.200.253:/exports/state/oleg410-client.virtnet on /var/lib/stateless/state type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.204.10,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg410-client.virtnet/boot on /boot type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.204.10,local_lock=none,addr=192.168.200.253) 192.168.200.253:/exports/state/oleg410-client.virtnet/etc/kdump.conf on /etc/kdump.conf type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.204.10,local_lock=none,addr=192.168.200.253) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=378180k,mode=700) 192.168.200.253:/exports/testreports/67594/testresults/sanityn-zfs-rocky8.10_x86_64-rocky8.10_x86_64-retry1 on /tmp/testlogs type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.204.10,local_lock=none,addr=192.168.200.253) /dev/vda on /usr/sbin/mount.lustre type squashfs (ro,relatime) Checking servers environments Checking clients oleg410-client.virtnet environments /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions loading modules on: 'oleg410-server' oleg410-server: oleg410-server.virtnet: executing load_modules_local oleg410-server: Loading modules from /home/green/git/lustre-release/lustre oleg410-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg410-server: detected 4 online CPUs by sysfs oleg410-server: MODOPTS_LIBCFS= oleg410-server: Force libcfs to create 2 CPU partitions Setup mgs, mdt, osts Start mds1: mount -t lustre -o localrecov lustre-mdt1/mdt1 /mnt/lustre-mds1 oleg410-server: oleg410-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg410-client: oleg410-server: ssh exited with exit code 1 Started lustre-MDT0000 Start ost1: mount -t lustre -o localrecov lustre-ost1/ost1 /mnt/lustre-ost1 seq.cli-lustre-OST0000-super.width=65536 oleg410-server: oleg410-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg410-client: oleg410-server: ssh exited with exit code 1 Started lustre-OST0000 Start ost2: mount -t lustre -o localrecov lustre-ost2/ost2 /mnt/lustre-ost2 seq.cli-lustre-OST0001-super.width=65536 oleg410-server: oleg410-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all pdsh@oleg410-client: oleg410-server: ssh exited with exit code 1 Started lustre-OST0001 Starting client: oleg410-client.virtnet: -o user_xattr,flock 192.168.204.110@tcp:/lustre /mnt/lustre Starting client oleg410-client.virtnet: -o user_xattr,flock 192.168.204.110@tcp:/lustre /mnt/lustre Started clients oleg410-client.virtnet: 192.168.204.110@tcp:/lustre on /mnt/lustre type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) Starting client: oleg410-client.virtnet: -o user_xattr,flock 192.168.204.110@tcp:/lustre /mnt/lustre2 Starting client oleg410-client.virtnet: -o user_xattr,flock 192.168.204.110@tcp:/lustre /mnt/lustre2 Started clients oleg410-client.virtnet: 192.168.204.110@tcp:/lustre on /mnt/lustre2 type lustre (rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose) Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff98c8c2e36000.idle_timeout=debug osc.lustre-OST0000-osc-ffff98c8c370c800.idle_timeout=debug osc.lustre-OST0001-osc-ffff98c8c2e36000.idle_timeout=debug osc.lustre-OST0001-osc-ffff98c8c370c800.idle_timeout=debug disable quota as required ost.OSS.ost_io.nrs_policies=tbf projid ost.OSS.ost_io.nrs_tbf_rule=start ext_proj_rw projid={100} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.290546 s, 57.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0445674 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0394754 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0607639 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.061615 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823422 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126401 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112337 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140956 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114973 s, 9.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161242 s, 6.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168079 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138232 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182817 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.2115 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.266019 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.265172 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0448958 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0518713 s, 20.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0424536 s, 24.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0649878 s, 16.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0577268 s, 18.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0676948 s, 15.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0411495 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0523562 s, 20.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336068 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0416484 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0398047 s, 26.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0496889 s, 21.1 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0507271 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0640869 s, 16.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0475941 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0587953 s, 17.8 MB/s removed '/mnt/lustre/d77kh.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77kh.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77kh.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.34909 s, 19.6 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.94198 s, 21.2 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77kh.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_proj_rw ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=tbf projid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w projid={100}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r projid={100}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.341236 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0573541 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0664082 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0756631 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0932127 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118593 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0944222 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129172 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10542 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136262 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145684 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165388 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175004 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.221273 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.225162 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.274795 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.28956 s, 3.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0552785 s, 19.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0413417 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194226 s, 54.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0358473 s, 29.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276385 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239421 s, 43.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254191 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299419 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0547754 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0615973 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0055675 s, 188 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073984 s, 142 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0685679 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0819374 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00422887 s, 248 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00515932 s, 203 MB/s removed '/mnt/lustre/d77kh.sanityn/nrs_r_oleg410-client.virtnet' removed '/mnt/lustre/d77kh.sanityn/nrs_w_oleg410-client.virtnet' removed directory '/mnt/lustre/d77kh.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.43991 s, 19.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.80147 s, 10.7 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77kh.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_policies=fifo PASS 77kh (148s) == sanityn test 77ki: Add rule with unsupported TBF type should fail for generic TBF ========================================================== 06:20:18 (1786357218) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_tbf_rule=start ext_fail projid={100}&opcode={ost_write} rate=20: Invalid argument oleg410-server: error: set_param: setting 'ost/OSS/ost_io/nrs_tbf_rule'='start ext_fail projid={100}&opcode={ost_write} rate=20': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_policies=fifo PASS 77ki (12s) == sanityn test 77kj: Verify nodemap support for NRS TBF rule ========================================================== 06:20:30 (1786357230) On MGS 192.168.204.110, active = nodemap.active=1 On MGS 192.168.204.110, TBF = ost.OSS.ost_io.nrs_policies=tbf nodemap ost.OSS.ost_io.nrs_tbf_rule=start ext_nm_rw nodemap={TBF} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.406523 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0552063 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0626232 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0893879 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131964 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158724 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133921 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173531 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146266 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187733 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193072 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213407 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206302 s, 5.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245263 s, 4.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211243 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.263099 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.281845 s, 3.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0420096 s, 25.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290972 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0469955 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.045494 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0359297 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0408071 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243568 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.057824 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0355983 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336237 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0704897 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0357359 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0713413 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0398237 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0400918 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0417848 s, 25.1 MB/s removed '/mnt/lustre/d77kj.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77kj.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77kj.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.522 s, 19.0 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.94869 s, 21.2 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77kj.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete On MGS 192.168.204.110, TBF.id = On MGS 192.168.204.110, active = nodemap.active=0 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.322077 s, 52.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0395954 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0511336 s, 20.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186771 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167762 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201545 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179469 s, 5.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178472 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198972 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208329 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216897 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194102 s, 5.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232023 s, 4.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.221416 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.21724 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.225372 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.257191 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.036441 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00992581 s, 106 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0478271 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0443294 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0437084 s, 24.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241856 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0685377 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0408102 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.051465 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304251 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0586196 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0623913 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0630894 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0812051 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0429701 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0669322 s, 15.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff98c8c370c800.lock_unused_count=2 removed '/mnt/lustre/d77kj.sanityn/nrs_r_oleg410-client.virtnet' removed '/mnt/lustre/d77kj.sanityn/nrs_w_oleg410-client.virtnet' removed directory '/mnt/lustre/d77kj.sanityn' not_under_control: 1 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.71405 s, 22.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.52828 s, 68.6 MB/s Read runtime is 3 s, speed is 33.333333 IOPS read rate (33.333333) should exceed limit (20 * 1) cleanup_tbf /mnt/lustre/d77kj.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_nm_rw ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=tbf nodemap+opcode oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_tbf_rule=start ext_w nodemap={TBF}&opcode={ost_write} rate=20: Invalid argument oleg410-server: error: set_param: setting 'ost/OSS/ost_io/nrs_tbf_rule'='start ext_w nodemap={TBF}&opcode={ost_write} rate=20': Invalid argument oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_tbf_rule=start ext_r nodemap={TBF}&opcode={ost_read} rate=10: Invalid argument oleg410-server: error: set_param: setting 'ost/OSS/ost_io/nrs_tbf_rule'='start ext_r nodemap={TBF}&opcode={ost_read} rate=10': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 On MGS 192.168.204.110, active = nodemap.active=1 On MGS 192.168.204.110, TBF = ost.OSS.ost_io.nrs_tbf_rule=start ext_w nodemap={TBF}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nodemap={TBF}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.341075 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0582163 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0539965 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0459654 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0563272 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0402637 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0867555 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0361279 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0634569 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101446 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112451 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110074 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199068 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179668 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220258 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.25329 s, 4.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157167 s, 66.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330605 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200093 s, 52.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356804 s, 29.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0364844 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284201 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0407773 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292911 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301986 s, 34.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0409872 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0445754 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0333635 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0491377 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0497131 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0489791 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0689152 s, 15.2 MB/s removed '/mnt/lustre/d77kj.sanityn/nrs_r_oleg410-client.virtnet' removed '/mnt/lustre/d77kj.sanityn/nrs_w_oleg410-client.virtnet' removed directory '/mnt/lustre/d77kj.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.08342 s, 20.6 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.77137 s, 10.7 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77kj.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_policies=fifo On MGS 192.168.204.110, TBF.id = On MGS 192.168.204.110, active = nodemap.active=0 PASS 77kj (136s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 06:22:46 (1786357366) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (5s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 06:22:51 (1786357371) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg410-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.19433 s, 328 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg410-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.0778 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (51s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 06:23:43 (1786357423) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.201273 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0525144 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0629631 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0346811 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0541023 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0683409 s, 15.3 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.067408 s, 15.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0752243 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0564822 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.06215 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0375154 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117569 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128126 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119635 s, 8.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff98c8c370c800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff98c8c370c800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110483 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136911 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133953 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171055 s, 61.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202974 s, 51.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330309 s, 31.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312365 s, 33.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0377206 s, 27.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0360784 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312183 s, 33.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381128 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033604 s, 31.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0422647 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.034976 s, 30.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0468849 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0350115 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290617 s, 36.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322706 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0369483 s, 28.4 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg410-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg410-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.14051 s, 20.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.96712 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.196089 s, 85.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302814 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290105 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103155 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0977977 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110372 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12071 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134027 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109174 s, 9.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117472 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138133 s, 7.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138359 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116253 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12443 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131415 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157877 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170998 s, 6.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263945 s, 39.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201488 s, 52.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264986 s, 39.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302789 s, 34.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336413 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028365 s, 37.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266074 s, 39.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0349462 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327952 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0357446 s, 29.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0368117 s, 28.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0456988 s, 22.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0383662 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0405022 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0348516 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0458775 s, 22.9 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg410-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg410-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' not_under_control: 0 cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.06084 s, 20.7 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.95589 s, 21.2 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (67s) == sanityn test 77o: Changing rank should not panic ====== 06:24:51 (1786357491) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (5s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 06:24:56 (1786357496) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (47s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 06:25:43 (1786357543) mds.MDS.mdt.nrs_policies=tbf oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg410-client: oleg410-server: ssh exited with exit code 36 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg410-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg410-client: oleg410-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (16s) == sanityn test 77r: Change type of tbf policy at run time ========================================================== 06:25:59 (1786357559) mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 276184 kill md_thread xargs PASS 77r (42s) == sanityn test 77s: Check TBF LRU shrinker ============== 06:26:41 (1786357601) jobid_name=%e.%p jobid_var=TEST_77s_JOBVAR mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start touch jobid={touch.*} rate=1000 mds.MDS.mdt.nrs_tbf_rule=start unlink jobid={unlink.*} rate=1000 touch jobid={touch.*} 1000, ref 9 touch jobid={touch.*} 1000, ref 0 unlink jobid={unlink.*} 1000, ref 9 unlink jobid={unlink.*} 1000, ref 0 mds.MDS.mdt.nrs_policies=fifo jobid_var=procname_uid jobid_name=%e.%u PASS 77s (23s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 06:27:04 (1786357624) oleg410-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device oleg410-server: error: set_param: setting 'ost/OSS/*/nrs_orr_quantum'='1': No such device pdsh@oleg410-client: oleg410-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (4s) == sanityn test 79: xattr: intent error ================== 06:27:08 (1786357628) File: /mnt/lustre/d79.sanityn Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115238826936067 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-08-10 06:27:09.000000000 -0400 Modify: 2026-08-10 06:27:09.000000000 -0400 Change: 2026-08-10 06:27:09.000000000 -0400 Birth: 2026-08-10 06:27:08.000000000 -0400 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (13s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 06:27:21 (1786357641) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (2s) == sanityn test 80b: Accessing directory during migration ========================================================== 06:27:23 (1786357643) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 06:27:24 (1786357644) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (2s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 06:27:26 (1786357646) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 06:27:27 (1786357647) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (2s) == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 06:27:29 (1786357649) - open/close 1656 (time 1786357661.76 total 10.00 last 165.55) total: 2500 open/close in 15.08 seconds: 165.76 ops/second snapshot_time 1786357696.918199287 secs.nsecs start_time 1786357650.160635514 secs.nsecs elapsed_time 46.757563773 secs.nsecs open 2505 samples [usecs] 128 9981 2389757 2672315285 close 2504 samples [usecs] 96 1931 695499 239364257 mknod 2501 samples [usecs] 475 9463 2146684 2191623878 unlink 2500 samples [usecs] 480 28659 2921539 9689513447 mkdir 2 samples [usecs] 1129 1522 2651 3591125 rmdir 2 samples [usecs] 3221 5130 8351 36691741 rename 2500 samples [usecs] 400 12301 6678745 26361364289 getattr 3761 samples [usecs] 6 947 117152 14124772 parallel_rename_file 2500 samples [usecs] 400 12301 6678745 26361364289 crossdir_rename 2500 samples [usecs] 400 12301 6678745 26361364289 readpage 30 samples [usecs] 2413 18665 278783 3233732033 PASS 81d (50s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 06:28:19 (1786357699) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7638 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7638 PASS 82 (3s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 06:28:22 (1786357702) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 06:28:23 (1786357703) fail_loc=0x8000060b PASS 84 (9s) == sanityn test 85: Lustre API root cache race =========== 06:28:32 (1786357712) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1786357712 Finishing test test1 at 1786357713 PASS 85 (3s) == sanityn test 90: open/create and unlink striped directory ========================================================== 06:28:35 (1786357715) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (2s) == sanityn test 91: chmod and unlink striped directory === 06:28:37 (1786357717) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 06:28:38 (1786357718) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 06:28:39 (1786357719) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6183 0x1827 0x240000400 1 6143 0x17ff 0x280000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6184 0x1828 0x240000400 1 6144 0x1800 0x280000400 rc1=0 and rc2=0 PASS 93 (10s) == sanityn test 94: signal vs CP callback race =========== 06:28:49 (1786357729) osc.lustre-OST0000-osc-ffff98c8c2e36000.idle_timeout=0 osc.lustre-OST0000-osc-ffff98c8c370c800.idle_timeout=0 osc.lustre-OST0001-osc-ffff98c8c2e36000.idle_timeout=0 osc.lustre-OST0001-osc-ffff98c8c370c800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0710062 s, 115 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 3.09085 s, 1.3 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 6389: 295483 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0315908 s, 130 kB/s osc.lustre-OST0000-osc-ffff98c8c2e36000.idle_timeout=debug osc.lustre-OST0000-osc-ffff98c8c370c800.idle_timeout=debug osc.lustre-OST0001-osc-ffff98c8c2e36000.idle_timeout=debug osc.lustre-OST0001-osc-ffff98c8c370c800.idle_timeout=debug PASS 94 (11s) == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 06:29:00 (1786357740) llite.lustre-ffff98c8c2e36000.fast_read=0 llite.lustre-ffff98c8c370c800.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0224391 s, 93.5 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff98c8c2e36000.fast_read=1 llite.lustre-ffff98c8c370c800.fast_read=1 PASS 95a (16s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 06:29:16 (1786357756) llite.lustre-ffff98c8c2e36000.fast_read=0 llite.lustre-ffff98c8c370c800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.00327538 s, 3.8 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff98c8c2e36000.fast_read=1 llite.lustre-ffff98c8c370c800.fast_read=1 PASS 95b (8s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 06:29:24 (1786357764) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 06:29:26 (1786357766) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0020011 s, 2.0 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (3s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 06:29:28 (1786357768) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0371149 s, 56.5 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 06:29:30 (1786357770) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0431676 s, 48.6 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (3s) == sanityn test 100e: DoM: read on open and file size ==== 06:29:33 (1786357773) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.00244471 s, 4.9 MB/s PASS 100e (3s) == sanityn test 101a: Discard DoM data on unlink ========= 06:29:36 (1786357776) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00286944 s, 1.4 MB/s PASS 101a (2s) == sanityn test 101b: Discard DoM data on rename ========= 06:29:39 (1786357779) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00347838 s, 1.2 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 06:29:41 (1786357781) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0028727 s, 1.4 MB/s PASS 101c (4s) == sanityn test 102: Test open by handle of unlinked file ========================================================== 06:29:45 (1786357785) file: /mnt/lustre/f102.sanityn fh_bytes: 32 fh_type: 151 fh_data: d1 0b 00 00 02 00 00 00 e4 10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 file's parent FID is [0:0x0:0x0] file FID is [0x200000bd1:0x10e4:0x0] access via mount point '/mnt/lustre2' - OK access after unlink - OK check_fhandle_syscalls test Passed! PASS 102 (3s) == sanityn test 103: Test size correctness with lockahead ========================================================== 06:29:48 (1786357788) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1786357789 lockahead_test: lockahead_test.c:1171: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1786357791 Finishing test test23 at 1786357792 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 06:29:56 (1786357796) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 06:29:57 (1786357797) fail_loc=0x416 PASS 105 (18s) == sanityn test 106a: Verify the btime via statx() ======= 06:30:15 (1786357815) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 06:30:16 (1786357816) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00641606 s, 160 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115238826938603 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-08-10 06:30:17.000000000 -0400 Modify: 2026-08-10 06:30:17.000000000 -0400 Change: 2026-08-10 06:30:17.000000000 -0400 Birth: 2026-08-10 06:30:16.000000000 -0400 /mnt/lustre/f106b.sanityn 144115238826938603 -rw-r--r-- 1 0 0 1786357816 1786357817 1786357817 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115238826938603 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-08-10 06:30:17.000000000 -0400 Modify: 2026-08-10 06:30:17.000000000 -0400 Change: 2026-08-10 06:30:17.000000000 -0400 Birth: 2026-08-10 06:30:16.000000000 -0400 1786357817 1024 1 PASS 106b (3s) == sanityn test 106c: Verify statx attributes mask ======= 06:30:19 (1786357819) PASS 106c (2s) == sanityn test 107a: Basic grouplock conflict =========== 06:30:21 (1786357821) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.100225 s, 105 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 6193 0x1831 0x240000400 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 First grouplock blocks second one PASS 107a (5s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 06:30:26 (1786357826) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.102432 s, 102 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 6151 0x1807 0x280000400 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7638 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (9s) == sanityn test 108a: lseek: parallel updates ============ 06:30:35 (1786357835) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.0031165 s, 3.2 MB/s llite.lustre-ffff98c8c2e36000.hybrid_io=0 llite.lustre-ffff98c8c370c800.hybrid_io=0 fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 4.22364 s, 2.0 MB/s PASS 108a (7s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 06:30:42 (1786357842) 192.168.204.110@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg410-client.virtnet /mnt/lustre (opts:) 192.168.204.110@tcp:/lustre /mnt/lustre2 lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg410-client.virtnet /mnt/lustre2 (opts:) Starting race between client mounts (duration 30) Iteration 0 Starting client: oleg410-client.virtnet: -o user_xattr,flock 192.168.204.110@tcp:/lustre /mnt/lustre Starting client: oleg410-client.virtnet: -o user_xattr,flock 192.168.204.110@tcp:/lustre /mnt/lustre2 192.168.204.110@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.110@tcp:/lustre /mnt/lustre2 lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg410-client.virtnet /mnt/lustre (opts:) Stopping client oleg410-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' loading modules on: 'oleg410-server' oleg410-server: oleg410-server.virtnet: executing load_modules_local oleg410-server: Loading modules from /home/green/git/lustre-release/lustre oleg410-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg410-server: detected 4 online CPUs by sysfs oleg410-server: MODOPTS_LIBCFS= oleg410-server: Force libcfs to create 2 CPU partitions Iteration 1 Starting client: oleg410-client.virtnet: -o user_xattr,flock 192.168.204.110@tcp:/lustre /mnt/lustre Starting client: oleg410-client.virtnet: -o user_xattr,flock 192.168.204.110@tcp:/lustre /mnt/lustre2 192.168.204.110@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.110@tcp:/lustre /mnt/lustre2 lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg410-client.virtnet /mnt/lustre (opts:) Stopping client oleg410-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' loading modules on: 'oleg410-server' oleg410-server: oleg410-server.virtnet: executing load_modules_local oleg410-server: Loading modules from /home/green/git/lustre-release/lustre oleg410-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg410-server: detected 4 online CPUs by sysfs oleg410-server: MODOPTS_LIBCFS= oleg410-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg410-client.virtnet: -o user_xattr,flock 192.168.204.110@tcp:/lustre /mnt/lustre2 Starting client: oleg410-client.virtnet: -o user_xattr,flock 192.168.204.110@tcp:/lustre /mnt/lustre 192.168.204.110@tcp:/lustre /mnt/lustre lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.110@tcp:/lustre /mnt/lustre2 lustre rw,flock,checksum,encrypt,lazystatfs,lruresize,nolock,statfs_project,nosync_on_close,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg410-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg410-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' mdt/mdt options: 'mdt_enable_flr_ec=1' loading modules on: 'oleg410-server' oleg410-server: oleg410-server.virtnet: executing load_modules_local oleg410-server: Loading modules from /home/green/git/lustre-release/lustre oleg410-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1048: echo: write error: Device or resource busy oleg410-server: detected 4 online CPUs by sysfs oleg410-server: MODOPTS_LIBCFS= oleg410-server: Force libcfs to create 2 CPU partitions Starting client: oleg410-client.virtnet: -o user_xattr,flock 192.168.204.110@tcp:/lustre /mnt/lustre Starting client: oleg410-client.virtnet: -o user_xattr,flock 192.168.204.110@tcp:/lustre /mnt/lustre2 PASS 109 (37s) == sanityn test 110: do not grant another lock on resend ========================================================== 06:31:19 (1786357879) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115238860488707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-08-10 06:31:20.000000000 -0400 Modify: 2026-08-10 06:31:20.000000000 -0400 Change: 2026-08-10 06:31:20.000000000 -0400 Birth: 2026-08-10 06:31:19.000000000 -0400 fail_loc=0 fail_val=0 Connected clients: oleg410-client.virtnet oleg410-client.virtnet PASS 110 (169s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 06:34:08 (1786358048) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 06:34:09 (1786358049) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 06:34:10 (1786358050) lustre-OST0000-osc-ffff98c8d85b7800 active. lustre-OST0001-osc-ffff98c8d85b7800 active. lustre-OST0000-osc-ffff98c8d85b7800 active. lustre-OST0001-osc-ffff98c8d85b7800 active. MGC192.168.204.110@tcp active. PASS 113 (2s) == sanityn test 114: implicit default LMV inherit ======== 06:34:12 (1786358052) SKIP: sanityn test_114 We need at least 2 MDTs for this test SKIP 114 (1s) == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 06:34:13 (1786358053) SKIP: sanityn test_115 ldiskfs only test SKIP 115 (1s) == sanityn test 116: DNE: Set default LMV layout from a remote client ========================================================== 06:34:14 (1786358054) SKIP: sanityn test_116 needs >= 2 MDTs SKIP 116 (1s) == sanityn test 121: trunc append race =================== 06:34:15 (1786358055) 0+1 records in 0+1 records out 16384 bytes (16 kB, 16 KiB) copied, 0.00369303 s, 4.4 MB/s fail_loc=0x80001436 1+0 records in 1+0 records out 20480 bytes (20 kB, 20 KiB) copied, 0.015966 s, 1.3 MB/s 0000000 0a79 0a79 0a79 0a79 0a79 0a79 0a79 0a79 * 0004000 0000 0000 0000 0000 0000 0000 0000 0000 * 000c800 0a79 0a79 0a79 0a79 0a79 0a79 0a79 0a79 * 0011800 PASS 121 (4s) == sanityn test 200: service remains healthy while able to process request ========================================================== 06:34:19 (1786358059) at_max=10 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=9 fail_loc=0x1e1 fail_val=20 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.0882 s, 0.2 kB/s4096 bytes (4.1 kB, 4.0 KiB) copied, 20.0908 s, 0.2 kB/s 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.0851 s, 0.2 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.0921 s, 0.2 kB/s 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.0872 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.0904 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.0965 s, 0.2 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1414 s, 0.1 kB/s4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1518 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1448 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1511 s, 0.1 kB/s4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1485 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1682 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1764 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.2466 s, 0.1 kB/s 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.2559 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.2485 s, 0.1 kB/s4096 bytes (4.1 kB, 4.0 KiB) copied, 60.2629 s, 0.1 kB/s at_max=600 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=116 oleg410-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff98c8d3f58800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff98c8d3f58800.ost_server_uuid in IDLE state after 0 sec PASS 200 (67s) cleanup: ====================================================== == sanityn test complete, duration 5901 sec ============== 06:35:27 (1786358127) === sanityn: start cleanup 06:35:27 (1786358127) === Stopping clients: oleg410-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg410-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 06:36:35 (1786358195) ===