-----============= acceptance-small: sanityn ============----- Thu Oct 23 08:51:09 EDT 2025 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg138-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 40a 102 skipping tests SLOW=no: 33a === sanityn: start setup 08:51:29 (1761223889) === Starting client oleg138-client.virtnet: -o user_xattr,flock 192.168.201.138@tcp:/lustre /mnt/lustre2 Mount client oleg138-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.201.138@tcp:/lustre /mnt/lustre2 Started clients oleg138-client.virtnet: 192.168.201.138@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) oleg138-client.virtnet: executing check_config_client /mnt/lustre oleg138-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg138-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff9c32c91b1800.idle_timeout=debug osc.lustre-OST0000-osc-ffff9c32d0df3800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9c32c91b1800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9c32d0df3800.idle_timeout=debug disable quota as required oleg138-server: oleg138-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 08:51:56 (1761223916) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210711 s, 49.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7558] == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 08:51:58 (1761223918) PASS 0 (7s) == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 08:52:05 (1761223925) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (6s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 08:52:11 (1761223931) -rw-r--r-- 1 root root 0 Oct 23 08:52 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (6s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 08:52:17 (1761223937) -rw-r--r-- 1 root root 0 Oct 23 08:52 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (5s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 08:52:22 (1761223942) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (5s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 08:52:27 (1761223947) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (5s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 08:52:32 (1761223952) total 1 -rwxrwxrwx 1 root root 0 Oct 23 08:52 f2a -rwxrwxrwx 1 root root 0 Oct 23 08:52 f2b total 1 -rwxrwxrwx 1 root root 0 Oct 23 08:52 f2a -rwxrwxrwx 1 root root 0 Oct 23 08:52 f2b running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.008486 s, 60.3 kB/s PASS 2e (6s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 08:52:38 (1761223958) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (2s) == sanityn test 2g: check blocks update on sync write ==== 08:52:40 (1761223960) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.256674 s, 8.2 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (6s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 08:52:46 (1761223966) PASS 3 (6s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 08:52:52 (1761223972) PASS 4 (7s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 08:52:59 (1761223979) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (6s) == sanityn test 6: remove of open file on other node ============================================================================ 08:53:05 (1761223985) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (7s) == sanityn test 7: remove of open directory on other node ======================================================================= 08:53:12 (1761223992) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (6s) == sanityn test 8: remove of open special file on other node ==================================================================== 08:53:18 (1761223998) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (6s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 08:53:25 (1761224005) PASS 9a (5s) == sanityn test 9b: append to striped sparse file ======== 08:53:31 (1761224011) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0476666 s, 22.0 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.0562478 s, 0.1 kB/s Data read (expecting 'foo'): foo PASS 9b (5s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 08:53:36 (1761224016) 1+0 records in 1+0 records out 1 byte copied, 0.00445733 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00706511 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0141567 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0172469 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0214679 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0117465 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0147007 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0163066 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0110128 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00964501 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00893988 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0148803 s, 0.1 kB/s PASS 10a (7s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 08:53:43 (1761224023) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.0114885 s, 267 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0551229 s, 74.3 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.00103407 s, 3.0 MB/s PASS 10b (6s) == sanityn test 11: execution of file opened for write should return error ============================================================== 08:53:49 (1761224029) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7558 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 300: /mnt/lustre2/d11/f: Text file busy PASS 11 (5s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 08:53:55 (1761224035) warning: '-runtime' deprecated, use '-t runtime' instead - link 438 (time 1761224046.94 total 10.04 last 43.63) - link 830 (time 1761224056.95 total 20.05 last 39.17) - link 1262 (time 1761224066.96 total 30.06 last 43.16) - link 1705 (time 1761224076.96 total 40.06 last 44.28) - link 2136 (time 1761224087.01 total 50.11 last 42.89) - link 2597 (time 1761224097.01 total 60.11 last 46.07) - link 3035 (time 1761224107.02 total 70.12 last 43.76) - link 3469 (time 1761224117.02 total 80.12 last 43.39) - link 3887 (time 1761224127.10 total 90.20 last 41.48) total: 4289 link in 100.02 seconds: 42.88 ops/second using seed 3326650209 running for 100 seconds - stat 8213 (time 1761224048 ; total 11 ; last 11) - stat 10000 (time 1761224050 ; total 13 ; last 2) - stat 19841 (time 1761224061 ; total 24 ; last 11) - stat 20000 (time 1761224061 ; total 24 ; last 0) - stat 30000 (time 1761224071 ; total 34 ; last 10) - stat 39667 (time 1761224082 ; total 45 ; last 11) - stat 40000 (time 1761224082 ; total 45 ; last 0) - stat 47500 (time 1761224093 ; total 56 ; last 11) - stat 50000 (time 1761224099 ; total 62 ; last 6) - stat 54571 (time 1761224110 ; total 73 ; last 11) - stat 58926 (time 1761224121 ; total 84 ; last 11) - stat 60000 (time 1761224123 ; total 86 ; last 2) - stat 64366 (time 1761224134 ; total 97 ; last 11) total: 66130 stats in 100 seconds: 661.299988 stats/second - unlinked 0 (time 1761224087 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile4289) error: No such file or directory total: 4288 unlinks in 97 seconds: 44.206184 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (21354) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (21356) - No such process PASS 12 (156s) == sanityn test 13: test directory page revocation ======= 08:56:30 (1761224190) f13 f13 PASS 13 (7s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 08:56:38 (1761224198) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7558 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4923: 22541 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (7s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 08:56:44 (1761224204) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4923: 23120 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (6s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 08:56:50 (1761224210) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 350: 23698 Terminated $DIR1/$tdir/sleep 60 PASS 14b (6s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 08:56:56 (1761224216) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 362: 24280 Terminated $DIR1/$tdir/sleep 60 PASS 14c (6s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 08:57:02 (1761224222) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 373: 24862 Terminated $DIR1/$tdir/sleep 60 PASS 14d (7s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 08:57:09 (1761224229) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky sleep 5 for ZFS MDS Waiting for MDT destroys to complete fallocate on zfs doesn't consume space STRIPECOUNT=2 ORIGFREE=7528448 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7528448KB free > 800000KB, increase MAXFREE (or reduce fs size) sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (42s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 08:57:51 (1761224271) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.349332 s, 24.0 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.381388 s, 22.0 MB/s Chance of close/open is 1 in 50 Seed set to 4274 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[0] 1761224274.942089 mapwrite 0x6be231 thru 0x6cd714 (0xf4e4 bytes) 000004[0] 1761224275.079721 read 0x2c8f58 thru 0x2d34ca (0xa573 bytes) 000005[1] 1761224275.106771 mapwrite 0x5c9311 thru 0x5cbbfe (0x28ee bytes) 000006[1] 1761224275.217494 mapwrite 0x8f72eb thru 0x900f3b (0x9c51 bytes) 000007[0] 1761224275.331752 mapread 0x1f7cc0 thru 0x1f8474 (0x7b5 bytes) 000008[0] 1761224275.365703 write 0x8fc69f thru 0x90a99c (0xe2fe bytes) truncating to largest ever: 0x579453 000010[1] 1761224275.422291 trunc from 0x90a99d to 0x579452 (0x39154a bytes) 000011[0] 1761224275.493461 write 0x8fbcaf thru 0x8fcbe0 (0xf32 bytes) 000012[0] 1761224275.528741 mapread 0x7c4f94 thru 0x7cb857 (0x68c4 bytes) 000013[1] 1761224275.598849 mapwrite 0x5ca80a thru 0x5d0252 (0x5a49 bytes) 000015[1] 1761224275.881041 write 0x8393fc thru 0x844b17 (0xb71c bytes) 000016[0] 1761224275.926198 mapread 0x378657 thru 0x37e9fb (0x63a5 bytes) 000018[1] 1761224276.011675 mapwrite 0x5ea9a6 thru 0x5f5c9c (0xb2f7 bytes) 000020[0] 1761224276.209433 mapwrite 0x3c64b9 thru 0x3d2574 (0xc0bc bytes) 000022[1] 1761224276.333943 mapread 0x58abd8 thru 0x58c5f4 (0x1a1d bytes) 000027[1] 1761224276.463523 trunc from 0x8fcbe1 to 0x4a82f7 (0x4548e9 bytes) 000028[0] 1761224276.498411 write 0x463fda thru 0x4729db (0xea02 bytes) 000030[0] 1761224276.572343 mapwrite 0x7765cf thru 0x7809ec (0xa41e bytes) 000033[0] 1761224276.702655 trunc from 0x7809ed to 0x2089fe (0x577fee bytes) 000035[0] 1761224276.805491 mapwrite 0x9de29f thru 0x9e8a4d (0xa7af bytes) 000036[0] 1761224276.912520 trunc from 0x9e8a4e to 0x426486 (0x5c25c7 bytes) 000038[1] 1761224276.952390 read 0x27e20d thru 0x27e32d (0x121 bytes) 000040[1] 1761224276.982586 write 0x487ef8 thru 0x4905be (0x86c7 bytes) 000042[1] 1761224276.996967 mapwrite 0x4f884b thru 0x5035a0 (0xad56 bytes) 000043[0] 1761224277.133164 mapwrite 0x338d95 thru 0x345c9f (0xcf0b bytes) 000044[1] 1761224277.296923 mapread 0x1135e2 thru 0x114624 (0x1043 bytes) 000046[0] 1761224277.362318 trunc from 0x5035a1 to 0x2ca316 (0x23928a bytes) 000049[0] 1761224277.467764 mapread 0x27b961 thru 0x28b274 (0xf914 bytes) 000050[1] 1761224277.505060 read 0x2aa415 thru 0x2b71ee (0xcdda bytes) 000054[1] 1761224277.618483 write 0x14b1b0 thru 0x150cb7 (0x5b08 bytes) 000055[1] 1761224277.674841 mapwrite 0x5f2498 thru 0x5fb707 (0x9270 bytes) truncating to largest ever: 0x887dff 000056[1] 1761224277.807500 trunc from 0x5fb708 to 0x887dfe (0x28c6f7 bytes) 000060[0] 1761224277.842080 trunc from 0x887dff to 0x1fb74a (0x68c6b4 bytes) 000061[1] 1761224277.908867 trunc from 0x1fb74b to 0x816820 (0x61b0d6 bytes) 000062[0] 1761224277.946814 mapread 0x784d96 thru 0x791c86 (0xcef1 bytes) truncating to largest ever: 0x94fb4b 000063[0] 1761224278.011930 trunc from 0x816821 to 0x94fb4a (0x13932a bytes) 000064[0] 1761224278.078801 read 0x424f57 thru 0x42d1ea (0x8294 bytes) 000066[0] 1761224278.098401 mapwrite 0x7a79d8 thru 0x7b2edb (0xb504 bytes) 000067[0] 1761224278.211398 write 0x9390d0 thru 0x93fe90 (0x6dc1 bytes) 000070[1] 1761224278.263438 trunc from 0x94fb4b to 0x83c3d8 (0x113772 bytes) 000074[1] 1761224278.409394 mapwrite 0x68b06d thru 0x6969c7 (0xb95b bytes) 000075[0] 1761224278.571464 read 0x4da6b2 thru 0x4e60f9 (0xba48 bytes) 000076[1] 1761224278.594795 read 0x2bd7a7 thru 0x2c0b34 (0x338e bytes) 000078[1] 1761224278.612594 mapread 0x192adc thru 0x19e86a (0xbd8f bytes) 000084[1] 1761224278.691186 trunc from 0x83c3d9 to 0x388bfd (0x4b37db bytes) 000085[1] 1761224278.738630 trunc from 0x388bfe to 0x6c4a38 (0x33be3b bytes) 000089[1] 1761224278.859680 read 0x3b7d92 thru 0x3bfefd (0x816c bytes) 000090[1] 1761224278.898755 mapread 0x54a83d thru 0x54d53e (0x2d02 bytes) 000092[1] 1761224278.950446 mapread 0x52b16a thru 0x52f4c1 (0x4358 bytes) 000095[1] 1761224279.021183 write 0x6cac3e thru 0x6d7847 (0xcc0a bytes) 000096[0] 1761224279.026883 write 0x9e0ba8 thru 0x9e4184 (0x35dd bytes) 000098[0] 1761224279.081607 mapwrite 0x3cfd1d thru 0x3d45d7 (0x48bb bytes) 000099[0] 1761224279.188297 write 0x4ad5be thru 0x4ae8fd (0x1340 bytes) 000102[1] 1761224279.297772 trunc from 0x9e4185 to 0x6c2d28 (0x32145c bytes) 000103[0] 1761224279.349271 mapread 0x027721 thru 0x035a4b (0xe32b bytes) 000104[0] 1761224279.407215 trunc from 0x6c2d29 to 0x10c0bb (0x5b6c6d bytes) 000105[1] 1761224279.476618 mapwrite 0x703984 thru 0x7081a1 (0x481e bytes) 000106[1] 1761224279.602399 read 0x02b104 thru 0x035925 (0xa822 bytes) 000108[0] 1761224279.621400 trunc from 0x7081a2 to 0x3d2e26 (0x33537b bytes) 000109[1] 1761224279.703492 write 0x1de5cb thru 0x1e1b28 (0x355e bytes) 000110[0] 1761224279.760497 write 0x8e9674 thru 0x8f0b1d (0x74aa bytes) 000114[0] 1761224279.848653 read 0x85062c thru 0x857572 (0x6f47 bytes) 000116[1] 1761224279.860103 write 0x481483 thru 0x48d03e (0xbbbc bytes) 000119[0] 1761224279.927719 mapread 0x8304d4 thru 0x83f255 (0xed82 bytes) 000122[1] 1761224280.003261 mapwrite 0x35effc thru 0x36c932 (0xd937 bytes) 000123[1] 1761224280.104056 trunc from 0x8f0b1e to 0x51ba57 (0x3d50c6 bytes) 000124[1] 1761224280.175662 mapread 0x2185ae thru 0x2191f3 (0xc46 bytes) 000125[0] 1761224280.208168 mapread 0x269083 thru 0x26e76d (0x56eb bytes) 000130[0] 1761224280.311927 trunc from 0x51ba58 to 0x621334 (0x1058dd bytes) 000131[0] 1761224280.377750 trunc from 0x621335 to 0x7cfe28 (0x1aeaf4 bytes) 000134[1] 1761224280.426055 trunc from 0x7cfe29 to 0x3997bd (0x43666b bytes) 000136[1] 1761224280.487103 read 0x397144 thru 0x3997bd (0x267a bytes) 000137[1] 1761224280.546013 mapwrite 0x64fcc9 thru 0x65e644 (0xe97c bytes) 000138[0] 1761224280.716648 mapread 0x2ea9de thru 0x2f2842 (0x7e65 bytes) 000139[1] 1761224280.796867 write 0x6beeb0 thru 0x6c841f (0x9570 bytes) 000140[1] 1761224280.820305 read 0x083416 thru 0x090905 (0xd4f0 bytes) 000144[0] 1761224280.849809 mapwrite 0x6eb299 thru 0x6ec962 (0x16ca bytes) 000145[0] 1761224280.980256 mapwrite 0x530ddc thru 0x54052d (0xf752 bytes) 000146[0] 1761224281.094497 trunc from 0x6ec963 to 0x1f66e0 (0x4f6282 bytes) 000149[0] 1761224281.142630 write 0x551a8c thru 0x55c664 (0xabd9 bytes) 000151[0] 1761224281.159199 read 0x2da283 thru 0x2e02c9 (0x6047 bytes) 000152[0] 1761224281.207105 mapwrite 0x6e380d thru 0x6e63ce (0x2bc2 bytes) 000153[0] 1761224281.331470 trunc from 0x6e63cf to 0x3b7986 (0x32ea48 bytes) 000159[0] 1761224281.403261 mapread 0x17f544 thru 0x18a0e1 (0xab9e bytes) 000163[0] 1761224281.473260 write 0x1613b0 thru 0x16e5a1 (0xd1f2 bytes) 000168[0] 1761224281.545737 mapwrite 0x3b860b thru 0x3c1fe6 (0x99dc bytes) 000171[0] 1761224281.673987 write 0x3c45d0 thru 0x3ce2ba (0x9ceb bytes) 000173[1] 1761224281.719522 mapwrite 0x1cef40 thru 0x1d7274 (0x8335 bytes) 000175[0] 1761224281.848811 write 0x2b15dc thru 0x2b5e76 (0x489b bytes) 000176[1] 1761224281.874473 trunc from 0x3ce2bb to 0x38dd1b (0x4059f bytes) 000180[1] 1761224281.989095 read 0x289683 thru 0x297b9d (0xe51b bytes) 000181[1] 1761224282.058473 trunc from 0x38dd1c to 0x0b05a3 (0x2dd778 bytes) 000185[0] 1761224282.119671 mapread 0x056a13 thru 0x057ff7 (0x15e5 bytes) 000187[1] 1761224282.153249 read 0x06484d thru 0x06e0f8 (0x98ac bytes) 000188[1] 1761224282.192232 write 0x8cfb51 thru 0x8d6302 (0x67b2 bytes) 000191[1] 1761224282.226753 mapwrite 0x9e7029 thru 0x9f50f2 (0xe0ca bytes) 000195[0] 1761224282.450378 read 0x24003e thru 0x247531 (0x74f4 bytes) 000196[1] 1761224282.491020 read 0x36b694 thru 0x3750ae (0x9a1b bytes) 000198[0] 1761224282.531807 mapwrite 0x4c2eaa thru 0x4c97e0 (0x6937 bytes) 000200[0] 1761224282.614628 trunc from 0x9f50f3 to 0x7c1312 (0x233de0 bytes) 000202[0] 1761224282.666927 mapread 0x6e04d6 thru 0x6eebaa (0xe6d5 bytes) 000203[1] 1761224282.697972 write 0x625457 thru 0x627963 (0x250d bytes) 000204[0] 1761224282.726958 read 0x4628ee thru 0x464dfe (0x2511 bytes) 000205[0] 1761224282.785903 write 0x2e58d2 thru 0x2f3de6 (0xe515 bytes) 000209[1] 1761224282.820735 mapread 0x2f7ab9 thru 0x306d7d (0xf2c5 bytes) 000210[0] 1761224282.888327 write 0x8c65f0 thru 0x8ccac2 (0x64d3 bytes) 000211[1] 1761224282.907316 mapwrite 0x8b596f thru 0x8c1515 (0xbba7 bytes) 000213[1] 1761224283.033214 mapwrite 0x889eb3 thru 0x8922fe (0x844c bytes) 000214[0] 1761224283.100601 mapwrite 0x682bb6 thru 0x68900a (0x6455 bytes) 000215[1] 1761224283.201734 write 0x01aacf thru 0x01f199 (0x46cb bytes) 000216[1] 1761224283.223640 mapread 0x00c343 thru 0x0169ec (0xa6aa bytes) 000217[1] 1761224283.252580 mapwrite 0x41d8d0 thru 0x4287f5 (0xaf26 bytes) 000218[0] 1761224283.332825 write 0x7e12ff thru 0x7ec7ec (0xb4ee bytes) 000219[0] 1761224283.353123 mapread 0x4b907e thru 0x4c5ab5 (0xca38 bytes) 000220[1] 1761224283.405560 mapread 0x6fef43 thru 0x70ecfb (0xfdb9 bytes) 000221[0] 1761224283.482390 write 0x593aa3 thru 0x59f75a (0xbcb8 bytes) 000222[0] 1761224283.523606 read 0x0c4312 thru 0x0c63ad (0x209c bytes) 000225[0] 1761224283.569281 trunc from 0x8ccac3 to 0x039892 (0x893230 bytes) 000226[0] 1761224283.620284 read 0x02b25d thru 0x032fd2 (0x7d76 bytes) 000227[0] 1761224283.641180 write 0x1b2d7e thru 0x1b33da (0x65d bytes) 000228[1] 1761224283.654279 trunc from 0x1b33db to 0x72d27e (0x579ea4 bytes) 000230[1] 1761224283.731092 write 0x3f380e thru 0x3fc632 (0x8e25 bytes) 000231[0] 1761224283.775841 mapwrite 0x5cef93 thru 0x5cf3ba (0x428 bytes) 000232[0] 1761224283.915099 write 0x282766 thru 0x28797d (0x5218 bytes) 000240[1] 1761224284.088521 mapread 0x25559c thru 0x25b8e9 (0x634e bytes) 000243[1] 1761224284.203933 trunc from 0x72d27f to 0x7203f0 (0xce8e bytes) 000246[0] 1761224284.267472 mapwrite 0x430c32 thru 0x43319b (0x256a bytes) 000247[1] 1761224284.363731 write 0x999357 thru 0x99f697 (0x6341 bytes) 000253[0] 1761224284.431887 trunc from 0x99f698 to 0x132a64 (0x86cc33 bytes) 000255[1] 1761224284.487210 mapwrite 0x19336c thru 0x19412c (0xdc1 bytes) 000257[1] 1761224284.612362 read 0x163a1e thru 0x172be3 (0xf1c6 bytes) 000258[0] 1761224284.655176 mapread 0x027e90 thru 0x0342e0 (0xc451 bytes) 000259[1] 1761224284.769172 write 0x9e2f04 thru 0x9e5a4d (0x2b4a bytes) 000260[1] 1761224284.779088 mapwrite 0x7d13dc thru 0x7d496a (0x358f bytes) 000263[0] 1761224284.946918 mapread 0x4ffe06 thru 0x505b0e (0x5d09 bytes) 000264[0] 1761224285.008272 mapread 0x2192f6 thru 0x226bd2 (0xd8dd bytes) 000265[1] 1761224285.102727 mapwrite 0x7580db thru 0x75a71a (0x2640 bytes) 000266[1] 1761224285.184465 read 0x6e400d thru 0x6e7582 (0x3576 bytes) 000268[0] 1761224285.191676 mapread 0x023379 thru 0x02fb81 (0xc809 bytes) 000269[1] 1761224285.274283 mapread 0x647419 thru 0x649d56 (0x293e bytes) 000271[1] 1761224285.309412 mapread 0x4cf268 thru 0x4dd5ec (0xe385 bytes) 000272[1] 1761224285.358854 mapread 0x87f938 thru 0x888980 (0x9049 bytes) 000274[1] 1761224285.375885 mapwrite 0x7de040 thru 0x7ed2c5 (0xf286 bytes) 000277[0] 1761224285.524562 mapread 0x6ffdc6 thru 0x703909 (0x3b44 bytes) 000278[1] 1761224285.570436 mapread 0x027c35 thru 0x02d53a (0x5906 bytes) 000279[1] 1761224285.607170 trunc from 0x9e5a4e to 0x213839 (0x7d2214 bytes) 000280[1] 1761224285.674061 mapwrite 0x25a10c thru 0x264505 (0xa3fa bytes) 000283[1] 1761224285.756202 mapwrite 0x6e2ccc thru 0x6f03b7 (0xd6ec bytes) 000285[0] 1761224285.842310 read 0x488b75 thru 0x49875e (0xfbea bytes) 000286[1] 1761224285.973826 mapwrite 0x934475 thru 0x9377ce (0x335a bytes) 000290[1] 1761224286.097636 mapread 0x3eca0c thru 0x3ed2b4 (0x8a9 bytes) 000292[0] 1761224286.126394 mapwrite 0x9d205a thru 0x9d7f67 (0x5f0e bytes) 000293[0] 1761224286.327396 mapwrite 0x08f179 thru 0x0908e1 (0x1769 bytes) 000296[1] 1761224286.429828 read 0x41d9dc thru 0x429e12 (0xc437 bytes) 000298[0] 1761224286.498243 mapwrite 0x9ec278 thru 0x9f7c2b (0xb9b4 bytes) 000299[1] 1761224286.609764 read 0x71df27 thru 0x725fab (0x8085 bytes) 000300[1] 1761224286.634852 trunc from 0x9f7c2c to 0x46b246 (0x58c9e5 bytes) 000304[1] 1761224286.723137 mapread 0x378eeb thru 0x37c5c8 (0x36de bytes) 000305[1] 1761224286.745437 mapread 0x1e173d thru 0x1e177a (0x3e bytes) 000306[0] 1761224286.748945 mapwrite 0x0e72d3 thru 0x0ecb92 (0x58c0 bytes) 000315[0] 1761224286.950385 read 0x0e5ceb thru 0x0ed47c (0x7792 bytes) 000316[1] 1761224286.970109 mapwrite 0x8c3ed6 thru 0x8c694b (0x2a76 bytes) 000317[0] 1761224287.051349 trunc from 0x8c694c to 0x4a4b68 (0x421de3 bytes) 000318[1] 1761224287.092215 mapread 0x093d09 thru 0x09915c (0x5454 bytes) 000321[1] 1761224287.148656 mapread 0x25505f thru 0x2560eb (0x108d bytes) 000323[0] 1761224287.181113 mapread 0x27a543 thru 0x288034 (0xdaf2 bytes) 000325[1] 1761224287.225168 read 0x171658 thru 0x17d147 (0xbaf0 bytes) 000326[0] 1761224287.239282 mapread 0x032814 thru 0x033562 (0xd4f bytes) 000328[1] 1761224287.287835 read 0x283c88 thru 0x2855f9 (0x1972 bytes) 000329[0] 1761224287.292330 mapread 0x2933e2 thru 0x296ae2 (0x3701 bytes) 000330[1] 1761224287.293024 write 0x327734 thru 0x32c201 (0x4ace bytes) 000331[0] 1761224287.342319 mapwrite 0x46d5e5 thru 0x477def (0xa80b bytes) 000332[0] 1761224287.460005 write 0x6305ce thru 0x63e064 (0xda97 bytes) 000333[0] 1761224287.477773 mapwrite 0x72667b thru 0x7364b7 (0xfe3d bytes) 000335[0] 1761224287.618277 mapwrite 0x24d0ce thru 0x252198 (0x50cb bytes) 000336[0] 1761224287.686870 write 0x57b3d2 thru 0x586e16 (0xba45 bytes) 000337[0] 1761224287.721977 read 0x6307f3 thru 0x63e348 (0xdb56 bytes) 000339[0] 1761224287.736526 trunc from 0x7364b8 to 0x3270c9 (0x40f3ee bytes) 000342[0] 1761224287.794402 mapwrite 0x5156b9 thru 0x5197eb (0x4133 bytes) 000343[0] 1761224287.882321 mapwrite 0x785312 thru 0x78ebee (0x98dd bytes) 000345[1] 1761224287.978959 read 0x4b45a1 thru 0x4bf410 (0xae70 bytes) 000347[1] 1761224288.035981 write 0x3a4793 thru 0x3af574 (0xade2 bytes) 000348[1] 1761224288.078636 read 0x2920f6 thru 0x2a0ed8 (0xede3 bytes) 000353[1] 1761224288.140871 read 0x16a324 thru 0x1772f2 (0xcfcf bytes) 000355[0] 1761224288.149886 trunc from 0x78ebef to 0x26bf40 (0x522cae bytes) 000357[1] 1761224288.231573 mapwrite 0x0bef8c thru 0x0c5a88 (0x6afd bytes) truncating to largest ever: 0x9d9148 000360[1] 1761224288.397058 trunc from 0x26bf41 to 0x9d9147 (0x76d207 bytes) 000361[1] 1761224288.456834 mapwrite 0x5920b3 thru 0x59bd0e (0x9c5c bytes) 000363[0] 1761224288.514486 trunc from 0x9d9148 to 0x60c67a (0x3ccacd bytes) 000364[1] 1761224288.549344 trunc from 0x60c67b to 0x939e18 (0x32d79e bytes) 000365[1] 1761224288.616195 write 0x86e04c thru 0x87518b (0x7140 bytes) 000366[0] 1761224288.638879 write 0x7cfdc6 thru 0x7d7e14 (0x804f bytes) 000372[1] 1761224288.763380 mapwrite 0x1e6372 thru 0x1f0c5a (0xa8e9 bytes) 000375[0] 1761224288.864823 write 0x16bbfe thru 0x17a35e (0xe761 bytes) 000376[1] 1761224288.905248 mapwrite 0x6f8508 thru 0x6f9be5 (0x16de bytes) 000377[1] 1761224289.003448 mapwrite 0x569e9f thru 0x5755f0 (0xb752 bytes) 000379[0] 1761224289.124025 mapread 0x1d763d thru 0x1ddaf3 (0x64b7 bytes) 000380[1] 1761224289.161342 write 0x9c59a4 thru 0x9d4155 (0xe7b2 bytes) 000381[1] 1761224289.180682 mapwrite 0x55f69c thru 0x5637b7 (0x411c bytes) 000382[1] 1761224289.251831 read 0x63b833 thru 0x64af27 (0xf6f5 bytes) 000383[1] 1761224289.287056 mapwrite 0x118c5c thru 0x1224cb (0x9870 bytes) 000388[0] 1761224289.431637 mapread 0x180f81 thru 0x189de0 (0x8e60 bytes) 000394[0] 1761224289.473669 mapwrite 0x10d2e3 thru 0x10fa81 (0x279f bytes) 000398[1] 1761224289.641076 mapread 0x5160f8 thru 0x518278 (0x2181 bytes) 000399[0] 1761224289.681964 mapread 0x6258e5 thru 0x627c0e (0x232a bytes) 000400[1] 1761224289.737339 read 0x23aed2 thru 0x247655 (0xc784 bytes) 000403[0] 1761224289.759161 trunc from 0x9d4156 to 0x0bec24 (0x915531 bytes) 000404[1] 1761224289.825095 mapread 0x09d947 thru 0x0a82bb (0xa975 bytes) 000405[0] 1761224289.884396 mapread 0x059e04 thru 0x068d5d (0xef5a bytes) 000407[0] 1761224289.960987 read 0x07db16 thru 0x07e93a (0xe25 bytes) 000408[0] 1761224290.015156 write 0x511149 thru 0x51ded8 (0xcd90 bytes) 000411[1] 1761224290.018137 mapread 0x3601d2 thru 0x367f5e (0x7d8d bytes) 000412[0] 1761224290.156264 mapread 0x30a4bc thru 0x316875 (0xc3ba bytes) 000415[1] 1761224290.263209 write 0x4707a4 thru 0x47e34b (0xdba8 bytes) 000421[0] 1761224290.327051 trunc from 0x51ded9 to 0x84e1be (0x3302e6 bytes) 000422[0] 1761224290.403364 write 0x98213e thru 0x98ecfc (0xcbbf bytes) 000423[1] 1761224290.405750 trunc from 0x98ecfd to 0x415100 (0x579bfc bytes) 000424[0] 1761224290.491994 trunc from 0x415101 to 0x492373 (0x7d273 bytes) 000425[1] 1761224290.563082 trunc from 0x492374 to 0x1654f5 (0x32ce7e bytes) 000426[0] 1761224290.646031 read 0x125685 thru 0x12f118 (0x9a94 bytes) 000427[1] 1761224290.682686 mapwrite 0x0c5117 thru 0x0d4faf (0xfe99 bytes) 000429[0] 1761224290.827208 read 0x09f277 thru 0x0a9660 (0xa3ea bytes) 000435[1] 1761224290.907979 write 0x4a4f86 thru 0x4a5027 (0xa2 bytes) 000437[0] 1761224290.909997 read 0x1070ff thru 0x109038 (0x1f3a bytes) 000438[1] 1761224290.939720 read 0x0573ca thru 0x061378 (0x9faf bytes) 000439[0] 1761224290.955083 mapread 0x34bdb1 thru 0x35416c (0x83bc bytes) 000440[1] 1761224291.149362 mapwrite 0x6eb66d thru 0x6f10de (0x5a72 bytes) 000442[1] 1761224291.344451 mapwrite 0x035268 thru 0x0427c1 (0xd55a bytes) 000444[0] 1761224291.435136 mapread 0x06ee38 thru 0x07e0a8 (0xf271 bytes) 000445[0] 1761224291.677398 read 0x48b342 thru 0x495b53 (0xa812 bytes) 000446[1] 1761224291.716444 write 0x498957 thru 0x49e986 (0x6030 bytes) 000447[1] 1761224291.739170 read 0x6bc93d thru 0x6c6023 (0x96e7 bytes) 000450[1] 1761224291.802396 trunc from 0x6f10df to 0x9ceec9 (0x2dddeb bytes) 000452[1] 1761224291.873045 mapwrite 0x19b67b thru 0x1a4b21 (0x94a7 bytes) 000458[0] 1761224292.077219 read 0x599cfa thru 0x5a9391 (0xf698 bytes) 000463[0] 1761224292.116420 write 0x555f8d thru 0x55cc4a (0x6cbe bytes) 000464[1] 1761224292.145420 mapwrite 0x4e9a9f thru 0x4f59e0 (0xbf42 bytes) 000465[0] 1761224292.250199 trunc from 0x9ceeca to 0x6e79cd (0x2e74fc bytes) 000469[1] 1761224292.401129 mapwrite 0x8374a5 thru 0x841eda (0xaa36 bytes) 000473[0] 1761224292.610106 mapread 0x6b4ce9 thru 0x6ba118 (0x5430 bytes) 000475[0] 1761224292.672258 read 0x417aa1 thru 0x418d57 (0x12b7 bytes) 000483[1] 1761224292.823357 mapwrite 0x983460 thru 0x98e7b3 (0xb354 bytes) 000484[0] 1761224292.994939 write 0x04cc61 thru 0x0508bc (0x3c5c bytes) 000485[0] 1761224293.026512 mapread 0x7f40ff thru 0x7f7aad (0x39af bytes) 000487[0] 1761224293.099984 mapwrite 0x2489e2 thru 0x25423e (0xb85d bytes) 000488[1] 1761224293.215490 trunc from 0x98e7b4 to 0x8be297 (0xd051c bytes) 000489[1] 1761224293.295307 mapread 0x6795e6 thru 0x67f2f1 (0x5d0c bytes) 000491[1] 1761224293.335721 mapwrite 0x3c8d15 thru 0x3d2568 (0x9854 bytes) 000492[0] 1761224293.446819 mapread 0x0d44ad thru 0x0df7a4 (0xb2f8 bytes) 000493[1] 1761224293.531303 mapread 0x022a15 thru 0x027438 (0x4a24 bytes) 000494[1] 1761224293.562225 trunc from 0x8be298 to 0x4cca13 (0x3f1884 bytes) 000495[1] 1761224293.628695 mapread 0x31fa10 thru 0x3247d8 (0x4dc9 bytes) 000496[0] 1761224293.677679 mapread 0x3d64fd thru 0x3e1a88 (0xb58c bytes) 000499[1] 1761224293.752786 mapwrite 0x74c7e8 thru 0x750a6c (0x4285 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 4293 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000004[0] 1761224294.468794 write 0x40b000 thru 0x41632a (0xb32b bytes) 000006[0] 1761224294.483656 write 0x2b6000 thru 0x2bab3a (0x4b3b bytes) 000008[1] 1761224294.524494 mapwrite 0x73c000 thru 0x74b371 (0xf372 bytes) truncating to largest ever: 0x27a237 000009[0] 1761224294.664162 trunc from 0x74b372 to 0x27a236 (0x4d113b bytes) 000012[1] 1761224294.700404 read_OD 0x148000 thru 0x14dfff (0x6000 bytes) 000013[0] 1761224294.722444 mapwrite 0x38e000 thru 0x392c2d (0x4c2e bytes) 000014[0] 1761224294.822747 read 00000000 thru 0x005aca (0x5acb bytes) 000015[0] 1761224294.867972 write 0x3ba000 thru 0x3c9172 (0xf173 bytes) 000017[1] 1761224294.870295 read_OD 0x342000 thru 0x342fff (0x1000 bytes) 000020[1] 1761224294.897067 read_OD 0x104000 thru 0x112fff (0xf000 bytes) 000023[1] 1761224294.903851 mapwrite 0x0c4000 thru 0x0cc60d (0x860e bytes) 000024[0] 1761224295.041737 write 0x958000 thru 0x9605e8 (0x85e9 bytes) 000025[0] 1761224295.094574 mapread 0x086000 thru 0x091109 (0xb10a bytes) 000026[0] 1761224295.165799 mapread 0x867000 thru 0x86b1f2 (0x41f3 bytes) 000027[0] 1761224295.202666 trunc from 0x9605e9 to 0x1570fc (0x8094ec bytes) 000028[0] 1761224295.219644 read 0x086000 thru 0x0895a5 (0x35a6 bytes) 000029[1] 1761224295.220485 read_OD 0x02b000 thru 0x032fff (0x8000 bytes) truncating to largest ever: 0x5b0bae 000031[1] 1761224295.261413 trunc from 0x1570fd to 0x5b0bad (0x459ab1 bytes) 000032[0] 1761224295.301603 read 0x339000 thru 0x3465a1 (0xd5a2 bytes) 000033[0] 1761224295.387898 mapread 0x48d000 thru 0x495fbd (0x8fbe bytes) 000034[1] 1761224295.413872 mapwrite 0x85a000 thru 0x86447c (0xa47d bytes) 000035[0] 1761224295.558073 write 0x35f000 thru 0x36c1c0 (0xd1c1 bytes) 000036[0] 1761224295.596147 read 0x03e000 thru 0x0438d0 (0x58d1 bytes) 000038[0] 1761224295.604064 trunc from 0x86447d to 0x2427d4 (0x621ca8 bytes) 000040[0] 1761224295.642981 mapwrite 0x1d4000 thru 0x1df944 (0xb945 bytes) truncating to largest ever: 0x8677ed 000041[1] 1761224295.730629 trunc from 0x2427d5 to 0x8677ec (0x625018 bytes) 000043[0] 1761224295.799703 trunc from 0x8677ed to 0x568567 (0x2ff285 bytes) 000044[1] 1761224295.833879 trunc from 0x568568 to 0x7820ec (0x219b85 bytes) 000047[0] 1761224295.912596 trunc from 0x7820ed to 0x38179e (0x40094e bytes) 000048[0] 1761224295.950954 mapwrite 0x826000 thru 0x829c93 (0x3c94 bytes) 000049[0] 1761224296.101364 mapread 0x22e000 thru 0x233cc8 (0x5cc9 bytes) 000050[1] 1761224296.144748 read_OD 0x601000 thru 0x608fff (0x8000 bytes) 000051[0] 1761224296.182803 write 0x8ab000 thru 0x8ab782 (0x783 bytes) 000052[0] 1761224296.193499 trunc from 0x8ab783 to 0x5011de (0x3aa5a4 bytes) 000053[0] 1761224296.230806 mapwrite 0x1dd000 thru 0x1e55dd (0x85de bytes) 000055[0] 1761224296.361876 mapwrite 0x0f7000 thru 0x106b74 (0xfb75 bytes) 000056[0] 1761224296.455235 mapwrite 0x5ba000 thru 0x5c34e7 (0x94e8 bytes) truncating to largest ever: 0x9cf6d4 000059[0] 1761224296.589350 trunc from 0x5c34e8 to 0x9cf6d3 (0x40c1ec bytes) 000060[0] 1761224296.614812 trunc from 0x9cf6d4 to 0x6e7d2e (0x2e79a5 bytes) 000061[1] 1761224296.641468 write_OD 0x717000 thru 0x71afff (0x4000 bytes) 000064[1] 1761224296.726388 read_OD 0x22a000 thru 0x234fff (0xb000 bytes) 000067[1] 1761224296.738425 trunc from 0x71b000 to 0x01f9d7 (0x6fb628 bytes) 000068[0] 1761224296.760158 write 0x6fc000 thru 0x6fdb6c (0x1b6d bytes) 000075[1] 1761224296.886890 read_OD 0x23d000 thru 0x23ffff (0x3000 bytes) 000078[1] 1761224296.922185 trunc from 0x6fdb6d to 0x736365 (0x387f9 bytes) 000080[0] 1761224296.952357 trunc from 0x736366 to 0x092e18 (0x6a354d bytes) 000081[0] 1761224296.987744 mapwrite 0x488000 thru 0x48fffe (0x7fff bytes) 000082[0] 1761224297.085275 mapread 0x427000 thru 0x431082 (0xa083 bytes) 000083[1] 1761224297.119015 mapread 0x196000 thru 0x19ce3e (0x6e3f bytes) 000087[1] 1761224297.172035 trunc from 0x48ffff to 0x786213 (0x2f6215 bytes) 000088[0] 1761224297.210383 trunc from 0x786214 to 0x511c2c (0x2745e7 bytes) 000089[1] 1761224297.244473 write_OD 0x1f2000 thru 0x1f6fff (0x5000 bytes) 000091[0] 1761224297.323486 trunc from 0x511c2d to 0x3ca42e (0x1477fe bytes) 000093[1] 1761224297.375424 read_OD 0x30f000 thru 0x313fff (0x5000 bytes) 000098[0] 1761224297.405604 read 0x109000 thru 0x10ab60 (0x1b61 bytes) 000099[0] 1761224297.505102 mapwrite 0x6e0000 thru 0x6ea939 (0xa93a bytes) 000100[1] 1761224297.704788 write_OD 0x291000 thru 0x29cfff (0xc000 bytes) 000101[0] 1761224297.764381 mapread 0x652000 thru 0x652b88 (0xb89 bytes) 000102[1] 1761224297.783324 trunc from 0x6ea93a to 0x843e75 (0x15953c bytes) 000104[0] 1761224297.835293 mapwrite 0x682000 thru 0x683c99 (0x1c9a bytes) 000106[1] 1761224297.899128 trunc from 0x843e76 to 0x68b5ec (0x1b8889 bytes) 000108[1] 1761224297.941426 write_OD 0x0b8000 thru 0x0bdfff (0x6000 bytes) 000109[0] 1761224297.993314 mapread 0x40b000 thru 0x415eaa (0xaeab bytes) 000110[1] 1761224298.059232 trunc from 0x68b5ed to 0x29cafa (0x3eeaf2 bytes) 000113[0] 1761224298.124939 write 0x525000 thru 0x533ef3 (0xeef4 bytes) 000119[0] 1761224298.220000 mapwrite 0x707000 thru 0x7156c9 (0xe6ca bytes) 000120[1] 1761224298.336631 mapwrite 0x480000 thru 0x4886a7 (0x86a8 bytes) 000121[0] 1761224298.512568 trunc from 0x7156ca to 0x81c774 (0x1070ab bytes) 000122[1] 1761224298.583030 write_OD 0x167000 thru 0x171fff (0xb000 bytes) 000124[0] 1761224298.658228 read 0x5c5000 thru 0x5d3882 (0xe883 bytes) 000125[0] 1761224298.675276 mapread 0x384000 thru 0x392de3 (0xede4 bytes) 000127[0] 1761224298.719323 mapwrite 0x09d000 thru 0x0a1cea (0x4ceb bytes) 000129[1] 1761224298.851639 mapwrite 0x346000 thru 0x350d65 (0xad66 bytes) 000131[0] 1761224299.031195 mapwrite 0x7e9000 thru 0x7f6980 (0xd981 bytes) 000132[1] 1761224299.148023 write_OD 0x4f4000 thru 0x4fbfff (0x8000 bytes) 000136[1] 1761224299.232892 read_OD 0x3dc000 thru 0x3e8fff (0xd000 bytes) 000139[0] 1761224299.249280 read 0x5ba000 thru 0x5c3723 (0x9724 bytes) 000140[0] 1761224299.260198 mapwrite 0x487000 thru 0x48d1be (0x61bf bytes) 000141[1] 1761224299.337989 write_OD 0x61f000 thru 0x628fff (0xa000 bytes) 000151[1] 1761224299.388961 mapwrite 0x4ad000 thru 0x4bcaf5 (0xfaf6 bytes) 000156[0] 1761224299.547501 read 0x274000 thru 0x27495a (0x95b bytes) 000158[0] 1761224299.567850 read 0x583000 thru 0x58a49d (0x749e bytes) 000159[0] 1761224299.576424 write 0x68e000 thru 0x68e143 (0x144 bytes) 000160[0] 1761224299.611112 trunc from 0x81c775 to 0x1229cb (0x6f9da9 bytes) 000161[0] 1761224299.650813 read 0x0ba000 thru 0x0c74b9 (0xd4ba bytes) 000168[1] 1761224299.770933 trunc from 0x1229cc to 0x08c6ba (0x96311 bytes) 000169[1] 1761224299.802214 trunc from 0x08c6bb to 0x0066de (0x85fdc bytes) 000170[0] 1761224299.821577 mapread 0x006000 thru 0x0063ae (0x3af bytes) 000171[0] 1761224299.836966 mapwrite 0x862000 thru 0x870dac (0xedad bytes) 000172[1] 1761224300.025632 trunc from 0x870dad to 0x87ead6 (0xdd2a bytes) 000173[0] 1761224300.075198 mapwrite 0x5cd000 thru 0x5d54f7 (0x84f8 bytes) 000175[1] 1761224300.187446 trunc from 0x87ead7 to 0x75fdff (0x11ecd7 bytes) 000178[1] 1761224300.305641 mapwrite 0x819000 thru 0x826f50 (0xdf51 bytes) 000181[1] 1761224300.412266 trunc from 0x826f51 to 0x6595d3 (0x1cd97d bytes) 000182[1] 1761224300.435987 trunc from 0x6595d4 to 0x6b4526 (0x5af53 bytes) 000183[0] 1761224300.451489 read 0x076000 thru 0x07eb02 (0x8b03 bytes) 000184[1] 1761224300.479157 trunc from 0x6b4527 to 0x96a6a4 (0x2b617e bytes) 000186[1] 1761224300.501230 trunc from 0x96a6a5 to 0x4924f7 (0x4d81ad bytes) 000187[0] 1761224300.510736 write 0x734000 thru 0x73ee86 (0xae87 bytes) 000188[1] 1761224300.535647 read_OD 0x4c8000 thru 0x4d6fff (0xf000 bytes) 000191[0] 1761224300.600120 write 0x344000 thru 0x34570c (0x170d bytes) 000197[0] 1761224300.696183 mapwrite 0x973000 thru 0x97e429 (0xb42a bytes) 000198[1] 1761224300.804129 mapread 0x2b6000 thru 0x2b7a68 (0x1a69 bytes) 000203[0] 1761224300.860843 trunc from 0x97e42a to 0x04326b (0x93b1be bytes) 000205[1] 1761224300.925086 mapread 0x003000 thru 0x003f61 (0xf62 bytes) 000206[0] 1761224300.963331 write 0x37b000 thru 0x382a12 (0x7a13 bytes) 000207[1] 1761224300.993669 mapread 0x229000 thru 0x232f74 (0x9f75 bytes) 000209[1] 1761224301.045594 read_OD 0x34f000 thru 0x359fff (0xb000 bytes) 000211[1] 1761224301.070337 mapwrite 0x47d000 thru 0x48cb96 (0xfb97 bytes) 000212[1] 1761224301.293223 trunc from 0x48cb97 to 0x4f9634 (0x6ca9e bytes) 000213[0] 1761224301.333319 write 0x7ad000 thru 0x7afec3 (0x2ec4 bytes) 000214[1] 1761224301.363190 write_OD 0x198000 thru 0x1a4fff (0xd000 bytes) 000215[1] 1761224301.443927 read_OD 0x6b1000 thru 0x6befff (0xe000 bytes) 000216[1] 1761224301.464315 trunc from 0x7afec4 to 0x58a7f7 (0x2256cc bytes) 000219[1] 1761224301.547258 write_OD 0x7fd000 thru 0x802fff (0x6000 bytes) truncating to largest ever: 0x9d9058 000222[1] 1761224301.598963 trunc from 0x803000 to 0x9d9057 (0x1d6058 bytes) 000226[0] 1761224301.674481 mapwrite 0x2d2000 thru 0x2df577 (0xd578 bytes) 000228[1] 1761224301.841731 trunc from 0x9d9058 to 0x67aea8 (0x35e1af bytes) 000229[1] 1761224301.889294 write_OD 0x26e000 thru 0x27bfff (0xe000 bytes) 000234[1] 1761224301.945702 read_OD 0x4ec000 thru 0x4f1fff (0x6000 bytes) 000236[1] 1761224301.952247 trunc from 0x67aea9 to 0x6e84c3 (0x6d61b bytes) 000239[1] 1761224302.024081 mapwrite 0x3a2000 thru 0x3accad (0xacae bytes) 000241[1] 1761224302.121976 read_OD 0x046000 thru 0x054fff (0xf000 bytes) 000242[1] 1761224302.148750 mapwrite 0x922000 thru 0x92f3e2 (0xd3e3 bytes) 000243[0] 1761224302.289113 write 0x68a000 thru 0x69609e (0xc09f bytes) 000248[1] 1761224302.337451 read_OD 0x0f4000 thru 0x0f9fff (0x6000 bytes) 000250[0] 1761224302.401496 mapread 0x3ad000 thru 0x3baead (0xdeae bytes) 000253[0] 1761224302.439780 write 0x96d000 thru 0x97b835 (0xe836 bytes) 000254[1] 1761224302.474354 mapread 0x0d7000 thru 0x0d7ed6 (0xed7 bytes) 000255[0] 1761224302.506958 read 0x0e9000 thru 0x0f74d6 (0xe4d7 bytes) 000256[1] 1761224302.529392 mapwrite 0x8ed000 thru 0x8f54f9 (0x84fa bytes) 000259[0] 1761224302.627780 mapread 0x95b000 thru 0x95fac8 (0x4ac9 bytes) 000260[0] 1761224302.672761 mapread 0x3ab000 thru 0x3b5e97 (0xae98 bytes) 000261[0] 1761224302.695943 write 0x08d000 thru 0x098f55 (0xbf56 bytes) 000264[1] 1761224302.774257 mapread 0x8bd000 thru 0x8bf9b1 (0x29b2 bytes) 000265[1] 1761224302.806753 mapread 0x612000 thru 0x616a37 (0x4a38 bytes) 000266[1] 1761224302.826846 read_OD 0x5aa000 thru 0x5affff (0x6000 bytes) 000267[1] 1761224302.834826 read_OD 0x3fb000 thru 0x408fff (0xe000 bytes) 000268[0] 1761224302.839679 read 0x355000 thru 0x362bce (0xdbcf bytes) 000270[0] 1761224302.856145 mapread 0x94b000 thru 0x959ff2 (0xeff3 bytes) 000271[0] 1761224302.885492 read 0x4d0000 thru 0x4d5b8a (0x5b8b bytes) 000274[1] 1761224302.897148 write_OD 0x907000 thru 0x911fff (0xb000 bytes) 000275[0] 1761224302.968260 mapwrite 0x8ea000 thru 0x8ed803 (0x3804 bytes) 000277[1] 1761224303.088361 write_OD 0x0fe000 thru 0x10bfff (0xe000 bytes) 000278[1] 1761224303.143971 write_OD 0x803000 thru 0x808fff (0x6000 bytes) 000283[0] 1761224303.214384 trunc from 0x97b836 to 0x23874f (0x7430e6 bytes) 000284[0] 1761224303.254424 mapwrite 0x93a000 thru 0x943f21 (0x9f22 bytes) 000293[1] 1761224303.379070 mapwrite 0x7d0000 thru 0x7d17ca (0x17cb bytes) 000294[1] 1761224303.444923 mapread 0x877000 thru 0x8822f7 (0xb2f8 bytes) 000297[0] 1761224303.489462 read 0x064000 thru 0x06748a (0x348b bytes) 000299[1] 1761224303.506850 read_OD 0x84c000 thru 0x854fff (0x9000 bytes) 000301[1] 1761224303.515595 trunc from 0x943f22 to 0x7bfd54 (0x1841cd bytes) 000302[0] 1761224303.559554 mapwrite 0x951000 thru 0x95afea (0x9feb bytes) 000303[0] 1761224303.647322 write 0x5d8000 thru 0x5e5931 (0xd932 bytes) 000304[0] 1761224303.675570 mapwrite 0x06b000 thru 0x07003f (0x5040 bytes) 000315[1] 1761224303.901740 mapwrite 0x7a4000 thru 0x7aae6b (0x6e6c bytes) 000319[1] 1761224304.005839 mapwrite 0x6d1000 thru 0x6d8848 (0x7849 bytes) 000320[0] 1761224304.122810 trunc from 0x95afeb to 0x25b966 (0x6ff684 bytes) 000321[1] 1761224304.160585 read_OD 0x024000 thru 0x02efff (0xb000 bytes) 000325[0] 1761224304.195203 mapread 0x008000 thru 0x016c53 (0xec54 bytes) 000331[0] 1761224304.213345 mapread 0x21e000 thru 0x228e59 (0xae5a bytes) 000333[0] 1761224304.236633 trunc from 0x25b967 to 0x041d5e (0x219c08 bytes) 000336[0] 1761224304.307217 write 0x976000 thru 0x985d0d (0xfd0e bytes) 000338[0] 1761224304.338605 read 0x97a000 thru 0x97c02c (0x202d bytes) 000339[1] 1761224304.338966 mapwrite 0x910000 thru 0x91779b (0x779c bytes) 000342[1] 1761224304.458451 trunc from 0x985d0e to 0x101db3 (0x883f5a bytes) 000344[1] 1761224304.476400 write_OD 0x539000 thru 0x53cfff (0x4000 bytes) 000345[0] 1761224304.522045 trunc from 0x53d000 to 0x84dea1 (0x310ea2 bytes) 000346[1] 1761224304.558413 mapwrite 0x912000 thru 0x92139a (0xf39b bytes) 000347[1] 1761224304.644862 mapread 0x0a9000 thru 0x0b1988 (0x8989 bytes) 000349[1] 1761224304.677073 mapwrite 0x925000 thru 0x92e3c7 (0x93c8 bytes) 000352[0] 1761224304.765496 mapread 0x91b000 thru 0x92142a (0x642b bytes) 000354[0] 1761224304.805433 read 0x3a7000 thru 0x3abddb (0x4ddc bytes) 000358[1] 1761224304.824821 mapwrite 0x646000 thru 0x64e5b7 (0x85b8 bytes) 000364[0] 1761224304.930886 mapread 0x4f7000 thru 0x4fb07c (0x407d bytes) 000365[0] 1761224304.966671 mapread 0x1a3000 thru 0x1a47ae (0x17af bytes) 000367[0] 1761224304.975651 read 0x09f000 thru 0x0a8179 (0x917a bytes) 000369[1] 1761224304.983817 trunc from 0x92e3c8 to 0x4e7859 (0x446b6e bytes) 000371[1] 1761224305.056754 trunc from 0x4e785a to 0x828dc1 (0x341568 bytes) 000373[0] 1761224305.111727 mapread 0x581000 thru 0x5868a5 (0x58a6 bytes) 000377[0] 1761224305.148261 mapwrite 0x404000 thru 0x40ae39 (0x6e3a bytes) 000378[0] 1761224305.229379 mapread 0x503000 thru 0x50bb1a (0x8b1b bytes) 000380[0] 1761224305.239783 trunc from 0x828dc2 to 0x651fc2 (0x1d6dff bytes) 000381[1] 1761224305.277053 mapwrite 0x405000 thru 0x406cb0 (0x1cb1 bytes) 000384[0] 1761224305.343359 mapread 0x3d8000 thru 0x3df0a1 (0x70a2 bytes) 000385[1] 1761224305.414979 trunc from 0x651fc3 to 0x4f7473 (0x15ab4f bytes) 000386[0] 1761224305.463734 trunc from 0x4f7474 to 0x7036b9 (0x20c246 bytes) 000388[0] 1761224305.497370 write 0x653000 thru 0x659134 (0x6135 bytes) 000389[0] 1761224305.503061 mapread 0x323000 thru 0x324673 (0x1674 bytes) 000391[1] 1761224305.532912 mapread 0x520000 thru 0x52aa4e (0xaa4f bytes) 000392[1] 1761224305.572098 mapwrite 0x9b3000 thru 0x9b85af (0x55b0 bytes) 000395[1] 1761224305.682599 write_OD 0x42c000 thru 0x430fff (0x5000 bytes) 000398[1] 1761224305.732863 mapread 0x753000 thru 0x757de7 (0x4de8 bytes) 000399[1] 1761224305.750379 mapread 0x25e000 thru 0x26c3c6 (0xe3c7 bytes) 000400[1] 1761224305.775330 read_OD 0x5a9000 thru 0x5adfff (0x5000 bytes) 000402[0] 1761224305.782996 mapread 0x0be000 thru 0x0c4e35 (0x6e36 bytes) 000403[0] 1761224305.811981 mapwrite 0x926000 thru 0x926a43 (0xa44 bytes) 000405[1] 1761224305.894666 read_OD 0x8a3000 thru 0x8affff (0xd000 bytes) 000406[0] 1761224305.915409 write 0x6d1000 thru 0x6e00ba (0xf0bb bytes) 000407[1] 1761224305.930220 write_OD 0x6e2000 thru 0x6e4fff (0x3000 bytes) 000408[1] 1761224306.007511 mapread 0x2f9000 thru 0x2fdba7 (0x4ba8 bytes) 000410[1] 1761224306.048912 read_OD 0x5fb000 thru 0x5fffff (0x5000 bytes) 000413[0] 1761224306.086229 mapread 0x41d000 thru 0x428916 (0xb917 bytes) 000417[1] 1761224306.112461 trunc from 0x9b85b0 to 0x037cb5 (0x9808fa bytes) 000418[0] 1761224306.172009 trunc from 0x037cb6 to 0x734e27 (0x6fd172 bytes) 000419[0] 1761224306.235288 write 0x961000 thru 0x9640af (0x30b0 bytes) 000420[1] 1761224306.240246 trunc from 0x9640b0 to 0x6038a4 (0x36080b bytes) 000421[0] 1761224306.289463 write 0x41d000 thru 0x42cfcb (0xffcc bytes) 000422[1] 1761224306.321515 trunc from 0x6038a5 to 0x63dd31 (0x3a48d bytes) 000423[0] 1761224306.389430 read 0x3f5000 thru 0x3fcfed (0x7fee bytes) 000424[0] 1761224306.415671 trunc from 0x63dd32 to 0x5d354e (0x6a7e3 bytes) 000426[1] 1761224306.466626 trunc from 0x5d354f to 0x18a424 (0x44912a bytes) 000427[0] 1761224306.503193 mapread 0x0fd000 thru 0x1023c1 (0x53c2 bytes) 000429[1] 1761224306.545861 trunc from 0x18a425 to 0x5752ea (0x3eaec6 bytes) 000431[1] 1761224306.574021 read_OD 0x29b000 thru 0x2a1fff (0x7000 bytes) 000434[1] 1761224306.601014 mapwrite 0x27e000 thru 0x283b44 (0x5b45 bytes) 000436[1] 1761224306.673723 mapread 0x44d000 thru 0x45819f (0xb1a0 bytes) 000437[0] 1761224306.704110 read 0x00c000 thru 0x01142d (0x542e bytes) 000440[1] 1761224306.736065 read_OD 0x1e4000 thru 0x1f1fff (0xe000 bytes) 000441[0] 1761224306.743736 mapwrite 0x956000 thru 0x956568 (0x569 bytes) 000443[0] 1761224306.827023 trunc from 0x956569 to 0x253ca0 (0x7028c8 bytes) 000445[0] 1761224306.882174 read 0x16c000 thru 0x1740b7 (0x80b8 bytes) 000446[0] 1761224306.929473 write 0x7fa000 thru 0x80187b (0x787c bytes) 000448[1] 1761224306.932978 trunc from 0x80187c to 0x5f42cc (0x20d5af bytes) 000449[0] 1761224306.970484 write 0x52b000 thru 0x533772 (0x8773 bytes) 000450[1] 1761224307.037573 write_OD 0x6ce000 thru 0x6cefff (0x1000 bytes) 000451[1] 1761224307.093440 write_OD 0x3fa000 thru 0x403fff (0xa000 bytes) 000452[1] 1761224307.137391 mapwrite 0x600000 thru 0x60c3d0 (0xc3d1 bytes) 000454[1] 1761224307.208848 trunc from 0x6cf000 to 0x6efc5f (0x20c60 bytes) 000456[0] 1761224307.250766 write 0x34a000 thru 0x35015e (0x615f bytes) 000457[1] 1761224307.289627 trunc from 0x6efc60 to 0x5c2688 (0x12d5d7 bytes) 000461[0] 1761224307.380827 write 0x15b000 thru 0x15efde (0x3fdf bytes) 000462[1] 1761224307.431049 read_OD 0x029000 thru 0x037fff (0xf000 bytes) 000463[0] 1761224307.464778 read 0x356000 thru 0x35ca64 (0x6a65 bytes) 000465[1] 1761224307.474851 mapwrite 0x397000 thru 0x39991a (0x291b bytes) 000471[1] 1761224307.626121 mapwrite 0x33c000 thru 0x34360e (0x760f bytes) 000473[1] 1761224307.712998 trunc from 0x5c2689 to 0x76962f (0x1a6fa7 bytes) 000482[0] 1761224308.006468 read 0x352000 thru 0x35aaa9 (0x8aaa bytes) 000483[0] 1761224308.115419 write 0x9dc000 thru 0x9ddf47 (0x1f48 bytes) 000484[0] 1761224308.137154 mapwrite 0x73e000 thru 0x749ae4 (0xbae5 bytes) 000487[1] 1761224308.261027 read_OD 0x0ed000 thru 0x0f3fff (0x7000 bytes) 000488[1] 1761224308.286264 write_OD 0x6d5000 thru 0x6d6fff (0x2000 bytes) 000489[1] 1761224308.335901 read_OD 0x034000 thru 0x03bfff (0x8000 bytes) 000492[0] 1761224308.345108 write 0x868000 thru 0x86acc0 (0x2cc1 bytes) 000495[0] 1761224308.392370 write 0x240000 thru 0x24661f (0x6620 bytes) 000497[0] 1761224308.397451 write 0x719000 thru 0x726dc0 (0xddc1 bytes) 000499[1] 1761224308.403177 write_OD 0x82a000 thru 0x838fff (0xf000 bytes) 000500[1] 1761224308.504649 write_OD 0x3eb000 thru 0x3effff (0x5000 bytes) All operations completed A-OK! PASS 16a (42s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 08:58:33 (1761224313) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.392068 s, 21.4 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.276029 s, 30.4 MB/s Chance of close/open is 1 in 50 Seed set to 4315 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read 000007[0] 1761224316.333403 mapwrite 0x2f2b7c thru 0x2f4507 (0x198c bytes) 000010[1] 1761224316.513471 mapwrite 0x8d3e06 thru 0x8d4b0f (0xd0a bytes) 000012[0] 1761224316.720879 write 0x37612c thru 0x377d6a (0x1c3f bytes) 000013[0] 1761224316.786331 read 0x0a3bac thru 0x0a489b (0xcf0 bytes) 000014[0] 1761224316.821129 read 0x48c8df thru 0x48d54b (0xc6d bytes) 000017[0] 1761224316.857934 mapread 0x89a823 thru 0x89baff (0x12dd bytes) 000018[0] 1761224316.932914 mapwrite 0x398ba2 thru 0x399703 (0xb62 bytes) 000020[1] 1761224317.035789 mapwrite 0x623661 thru 0x624156 (0xaf6 bytes) truncating to largest ever: 0x72f3de 000029[0] 1761224317.424991 trunc from 0x8d4b10 to 0x72f3dd (0x1a5732 bytes) 000030[1] 1761224317.510362 mapwrite 0x3b7bad thru 0x3b965e (0x1ab2 bytes) 000032[1] 1761224317.652083 read 0x619efb thru 0x61b6d0 (0x17d6 bytes) 000036[0] 1761224317.685235 write 0x10b810 thru 0x10d2d8 (0x1ac9 bytes) 000039[1] 1761224317.724819 read 0x18b4f3 thru 0x18b772 (0x280 bytes) 000040[0] 1761224317.745214 mapwrite 0x6727f0 thru 0x673b5d (0x136e bytes) 000041[0] 1761224317.834956 read 0x6da491 thru 0x6dbf72 (0x1ae2 bytes) 000042[0] 1761224317.843887 read 0x2bf4bc thru 0x2c0525 (0x106a bytes) 000043[0] 1761224317.851634 write 0x5b9826 thru 0x5baa30 (0x120b bytes) truncating to largest ever: 0x80c60a 000044[1] 1761224317.884765 trunc from 0x72f3de to 0x80c609 (0xdd22c bytes) 000045[1] 1761224317.975181 mapwrite 0x5d51a3 thru 0x5d62ad (0x110b bytes) 000047[0] 1761224318.121080 trunc from 0x80c60a to 0x6c4681 (0x147f88 bytes) 000048[1] 1761224318.177916 mapread 0x2b8249 thru 0x2b847a (0x232 bytes) 000053[0] 1761224318.281186 write 0x0e056e thru 0x0e0f2b (0x9be bytes) 000055[0] 1761224318.309908 trunc from 0x6c4682 to 0x71a426 (0x55da5 bytes) 000057[0] 1761224318.416214 mapwrite 0x5855cc thru 0x585cca (0x6ff bytes) 000062[0] 1761224318.555508 mapwrite 0x615126 thru 0x615621 (0x4fc bytes) 000066[1] 1761224318.674972 mapread 0x480c32 thru 0x481fb0 (0x137f bytes) 000068[0] 1761224318.707138 trunc from 0x71a427 to 0x42ccb7 (0x2ed76f bytes) 000070[0] 1761224318.773845 trunc from 0x42ccb8 to 0x1ced6b (0x25df4c bytes) 000071[0] 1761224318.813302 mapread 0x1b0fc4 thru 0x1b2575 (0x15b2 bytes) 000072[1] 1761224318.855613 read 0x17c744 thru 0x17cdbd (0x67a bytes) 000073[1] 1761224318.924405 mapread 0x06328d thru 0x0640a9 (0xe1d bytes) 000074[0] 1761224318.931099 trunc from 0x1ced6c to 0x6a84f8 (0x4d978d bytes) 000075[0] 1761224318.993573 read 0x124df3 thru 0x126788 (0x1996 bytes) 000076[1] 1761224319.015251 mapread 0x448889 thru 0x44a241 (0x19b9 bytes) 000077[0] 1761224319.079802 read 0x149796 thru 0x14a397 (0xc02 bytes) 000079[1] 1761224319.089474 read 0x454711 thru 0x454acd (0x3bd bytes) 000080[1] 1761224319.097372 mapwrite 0x0ce29e thru 0x0ceac7 (0x82a bytes) 000082[0] 1761224319.258479 trunc from 0x6a84f9 to 0x2b9c03 (0x3ee8f5 bytes) 000083[1] 1761224319.346227 trunc from 0x2b9c04 to 0x0cd529 (0x1ec6da bytes) 000087[1] 1761224319.456711 write 0x6d7728 thru 0x6d7dac (0x685 bytes) 000088[1] 1761224319.459206 trunc from 0x6d7dad to 0x7c386d (0xebac1 bytes) 000089[0] 1761224319.491354 write 0x534250 thru 0x5353c4 (0x1175 bytes) 000091[1] 1761224319.569738 mapread 0x2f2003 thru 0x2f2fab (0xfa9 bytes) 000093[1] 1761224319.613426 write 0x19ba13 thru 0x19caa8 (0x1096 bytes) 000096[1] 1761224319.664916 trunc from 0x7c386e to 0x3bb138 (0x408735 bytes) 000098[0] 1761224319.721235 trunc from 0x3bb139 to 0x181da9 (0x23938f bytes) 000099[1] 1761224319.792901 read 0x095704 thru 0x09586e (0x16b bytes) 000102[0] 1761224319.884233 mapread 0x035327 thru 0x0363e8 (0x10c2 bytes) 000104[1] 1761224319.904940 trunc from 0x181daa to 0x0a4479 (0xdd930 bytes) 000106[1] 1761224319.986720 trunc from 0x0a447a to 0x3ccd17 (0x32889e bytes) 000107[1] 1761224320.028653 mapwrite 0x22220c thru 0x222aee (0x8e3 bytes) 000109[0] 1761224320.185957 write 0x0375e0 thru 0x0376a9 (0xca bytes) truncating to largest ever: 0x886286 000113[0] 1761224320.310262 trunc from 0x3ccd18 to 0x886285 (0x4b956e bytes) 000115[0] 1761224320.373041 mapread 0x2e01f6 thru 0x2e0fbe (0xdc9 bytes) 000116[1] 1761224320.381678 read 0x443f11 thru 0x445c6e (0x1d5e bytes) 000123[0] 1761224320.430328 mapread 0x22e84f thru 0x22fd47 (0x14f9 bytes) 000125[1] 1761224320.457074 read 0x505835 thru 0x50604b (0x817 bytes) 000127[1] 1761224320.463080 mapread 0x76a26b thru 0x76ad5c (0xaf2 bytes) 000128[1] 1761224320.486967 mapwrite 0x0c8ce2 thru 0x0c90d4 (0x3f3 bytes) 000129[0] 1761224320.549078 mapwrite 0x181c87 thru 0x181df2 (0x16c bytes) 000130[0] 1761224320.616868 mapread 0x2e0553 thru 0x2e0fb7 (0xa65 bytes) 000132[0] 1761224320.665180 read 0x2425a7 thru 0x242be4 (0x63e bytes) truncating to largest ever: 0x90d081 000133[0] 1761224320.683602 trunc from 0x886286 to 0x90d080 (0x86dfb bytes) 000134[1] 1761224320.751138 mapread 0x390209 thru 0x391379 (0x1171 bytes) 000135[1] 1761224320.807287 read 0x3e13fd thru 0x3e2d07 (0x190b bytes) 000136[0] 1761224320.851228 trunc from 0x90d081 to 0x45c726 (0x4b095a bytes) 000139[0] 1761224320.914821 write 0x6d2a68 thru 0x6d3a95 (0x102e bytes) 000140[0] 1761224320.916082 write 0x456578 thru 0x457d94 (0x181d bytes) 000141[0] 1761224320.936192 mapread 0x54da99 thru 0x54fa4f (0x1fb7 bytes) 000145[0] 1761224320.983116 mapwrite 0x56dd3d thru 0x56e81b (0xadf bytes) truncating to largest ever: 0x96b39b 000147[1] 1761224321.065207 trunc from 0x6d3a96 to 0x96b39a (0x297905 bytes) 000150[0] 1761224321.184775 mapread 0x932708 thru 0x9341d3 (0x1acc bytes) 000151[1] 1761224321.226807 mapread 0x041350 thru 0x0419de (0x68f bytes) 000154[1] 1761224321.257188 read 0x446184 thru 0x446ef6 (0xd73 bytes) 000157[1] 1761224321.288456 read 0x774725 thru 0x776030 (0x190c bytes) 000158[0] 1761224321.300780 mapwrite 0x1514a1 thru 0x151f90 (0xaf0 bytes) 000161[1] 1761224321.467368 mapread 0x557f45 thru 0x558cec (0xda8 bytes) 000162[1] 1761224321.512718 read 0x88b6b8 thru 0x88b8d3 (0x21c bytes) 000163[0] 1761224321.577180 read 0x752da8 thru 0x753851 (0xaaa bytes) 000164[0] 1761224321.609601 write 0x2e3e3b thru 0x2e4734 (0x8fa bytes) 000165[0] 1761224321.724811 read 0x4a0037 thru 0x4a0f6d (0xf37 bytes) 000170[0] 1761224321.780191 trunc from 0x96b39b to 0x565bd3 (0x4057c7 bytes) 000171[0] 1761224321.829866 mapwrite 0x17167a thru 0x172b95 (0x151c bytes) 000172[0] 1761224321.918912 read 0x3d40be thru 0x3d46a4 (0x5e7 bytes) 000173[0] 1761224321.925475 write 0x7626ff thru 0x763760 (0x1062 bytes) 000174[0] 1761224321.926735 mapwrite 0x65f8bf thru 0x660d60 (0x14a2 bytes) 000177[0] 1761224322.003249 write 0x788221 thru 0x7894df (0x12bf bytes) 000179[1] 1761224322.081324 read 0x00c3a0 thru 0x00d738 (0x1399 bytes) 000180[1] 1761224322.133613 mapread 0x1298eb thru 0x129c98 (0x3ae bytes) 000181[0] 1761224322.157009 write 0x885dea thru 0x887157 (0x136e bytes) 000182[0] 1761224322.199342 read 0x62e3b1 thru 0x62faf6 (0x1746 bytes) 000184[1] 1761224322.224105 trunc from 0x887158 to 0x90ecc5 (0x87b6e bytes) 000185[0] 1761224322.296902 trunc from 0x90ecc6 to 0x77a294 (0x194a31 bytes) 000187[0] 1761224322.346298 write 0x0f6604 thru 0x0f7f1e (0x191b bytes) 000188[1] 1761224322.357731 read 0x262f4f thru 0x264917 (0x19c9 bytes) 000189[0] 1761224322.396639 write 0x769687 thru 0x769b0b (0x485 bytes) 000192[0] 1761224322.419174 trunc from 0x77a295 to 0x41215e (0x368136 bytes) 000194[0] 1761224322.468432 read 0x387aed thru 0x387d8d (0x2a1 bytes) 000196[1] 1761224322.510228 trunc from 0x41215f to 0x3ca2d8 (0x47e86 bytes) 000198[1] 1761224322.672425 trunc from 0x3ca2d9 to 0x4df93f (0x115667 bytes) 000199[0] 1761224322.693589 write 0x9a993d thru 0x9aaf83 (0x1647 bytes) 000202[0] 1761224322.802453 mapread 0x7ff661 thru 0x80120f (0x1baf bytes) 000203[1] 1761224322.869232 write 0x847e61 thru 0x8490ed (0x128d bytes) 000205[0] 1761224322.922290 mapread 0x080dd8 thru 0x0829bf (0x1be8 bytes) 000207[0] 1761224323.015597 write 0x06a16c thru 0x06b441 (0x12d6 bytes) 000210[0] 1761224323.111678 mapread 0x42ea79 thru 0x430857 (0x1ddf bytes) 000211[1] 1761224323.130687 mapwrite 0x219b81 thru 0x21addd (0x125d bytes) 000212[0] 1761224323.211501 write 0x705bdf thru 0x706ff4 (0x1416 bytes) 000214[0] 1761224323.257636 write 0x1fcff0 thru 0x1fde5b (0xe6c bytes) 000215[0] 1761224323.281498 mapread 0x4746c7 thru 0x475f4c (0x1886 bytes) 000216[0] 1761224323.317931 mapread 0x4e0b7f thru 0x4e263a (0x1abc bytes) 000220[0] 1761224323.377794 mapwrite 0x2d2ed4 thru 0x2d4932 (0x1a5f bytes) 000222[1] 1761224323.555432 mapread 0x13bf92 thru 0x13c4f4 (0x563 bytes) 000224[0] 1761224323.608665 mapwrite 0x61750e thru 0x6176c8 (0x1bb bytes) 000226[0] 1761224323.682615 mapwrite 0x8ea066 thru 0x8eb668 (0x1603 bytes) 000227[0] 1761224323.738331 mapwrite 0x804970 thru 0x805468 (0xaf9 bytes) 000229[1] 1761224323.800813 mapread 0x65f11d thru 0x6604ca (0x13ae bytes) 000230[0] 1761224323.832196 write 0x1ae126 thru 0x1aea2f (0x90a bytes) 000231[0] 1761224323.850081 write 0x82d15b thru 0x82e541 (0x13e7 bytes) 000236[1] 1761224323.895640 write 0x7747b2 thru 0x775013 (0x862 bytes) 000237[1] 1761224323.918295 write 0x7ef3ef thru 0x7ef605 (0x217 bytes) 000239[1] 1761224323.941188 write 0x89470e thru 0x894c47 (0x53a bytes) 000240[0] 1761224323.982062 write 0x4cf708 thru 0x4cf85b (0x154 bytes) 000241[0] 1761224324.038031 mapwrite 0x6dae8a thru 0x6dc37b (0x14f2 bytes) 000242[1] 1761224324.114458 trunc from 0x9aaf84 to 0x0113d3 (0x999bb0 bytes) 000243[0] 1761224324.171624 write 0x328321 thru 0x32924a (0xf2a bytes) 000244[0] 1761224324.207416 read 0x020f7d thru 0x02171d (0x7a1 bytes) 000246[0] 1761224324.273434 mapread 0x128e2f thru 0x129716 (0x8e8 bytes) 000247[1] 1761224324.296853 write 0x4d8281 thru 0x4d9916 (0x1696 bytes) 000249[1] 1761224324.331238 trunc from 0x4d9917 to 0x46c32f (0x6d5e7 bytes) 000252[0] 1761224324.378087 write 0x93fbe1 thru 0x93fe57 (0x277 bytes) 000253[1] 1761224324.407598 mapread 0x1ea6f4 thru 0x1eb7d7 (0x10e4 bytes) 000254[0] 1761224324.450215 read 0x495aa3 thru 0x495ecf (0x42d bytes) 000255[1] 1761224324.470116 write 0x6a4367 thru 0x6a5ac9 (0x1763 bytes) 000256[1] 1761224324.505567 mapread 0x3798e3 thru 0x37b480 (0x1b9e bytes) 000257[0] 1761224324.527944 mapwrite 0x4cb939 thru 0x4cceae (0x1576 bytes) 000259[0] 1761224324.622096 trunc from 0x93fe58 to 0x0eaa91 (0x8553c6 bytes) 000260[0] 1761224324.694819 mapread 0x01b959 thru 0x01bc6d (0x315 bytes) 000262[1] 1761224324.748778 mapwrite 0x97cd45 thru 0x97e210 (0x14cc bytes) 000263[1] 1761224324.865661 trunc from 0x97e211 to 0x2ddebf (0x6a0351 bytes) 000265[0] 1761224324.904196 write 0x317bd6 thru 0x3185a1 (0x9cc bytes) 000267[1] 1761224324.956256 trunc from 0x3185a2 to 0x40ec5f (0xf66be bytes) 000269[1] 1761224325.044331 write 0x3a87f8 thru 0x3a9ab2 (0x12bb bytes) 000270[1] 1761224325.066609 read 0x32572a thru 0x32654e (0xe25 bytes) 000271[1] 1761224325.081399 read 0x0bb325 thru 0x0bbbb2 (0x88e bytes) 000273[1] 1761224325.124701 mapwrite 0x046616 thru 0x04698d (0x378 bytes) 000277[1] 1761224325.312721 mapread 0x06b5cc thru 0x06b5f4 (0x29 bytes) 000278[1] 1761224325.344941 read 0x3dbc41 thru 0x3dd5f5 (0x19b5 bytes) 000281[0] 1761224325.420092 read 0x2eacdf thru 0x2eb226 (0x548 bytes) 000282[1] 1761224325.462196 write 0x52b2d3 thru 0x52c68b (0x13b9 bytes) 000285[1] 1761224325.519961 write 0x5ffe93 thru 0x6009cf (0xb3d bytes) 000286[0] 1761224325.538096 trunc from 0x6009d0 to 0x472c1b (0x18ddb4 bytes) 000293[1] 1761224325.629308 trunc from 0x472c1c to 0x7b90b5 (0x34649a bytes) 000295[0] 1761224325.697388 read 0x4d28e4 thru 0x4d2d8c (0x4a9 bytes) 000296[1] 1761224325.756951 read 0x743d2f thru 0x745b22 (0x1df4 bytes) 000297[1] 1761224325.792670 trunc from 0x7b90b6 to 0x59ac60 (0x21e455 bytes) 000298[0] 1761224325.893989 trunc from 0x59ac61 to 0x12d356 (0x46d90a bytes) 000299[1] 1761224325.966219 mapwrite 0x600b07 thru 0x600fac (0x4a6 bytes) 000301[0] 1761224326.072776 mapread 0x5c1177 thru 0x5c23dc (0x1266 bytes) 000303[1] 1761224326.141690 write 0x01c0d4 thru 0x01d7c6 (0x16f3 bytes) 000304[1] 1761224326.159608 write 0x19530b thru 0x195f97 (0xc8d bytes) 000307[0] 1761224326.166759 mapread 0x06b0e0 thru 0x06bf5a (0xe7b bytes) 000308[0] 1761224326.230096 write 0x24328f thru 0x2439aa (0x71c bytes) 000309[0] 1761224326.259578 write 0x02fb42 thru 0x030b67 (0x1026 bytes) 000314[0] 1761224326.404400 mapread 0x50144f thru 0x501f53 (0xb05 bytes) 000315[1] 1761224326.409850 mapwrite 0x67b9d7 thru 0x67bd60 (0x38a bytes) 000321[0] 1761224326.523310 trunc from 0x67bd61 to 0x2395d1 (0x44278f bytes) 000323[0] 1761224326.608331 write 0x4f724e thru 0x4f7332 (0xe5 bytes) 000325[0] 1761224326.615006 mapwrite 0x8d51b4 thru 0x8d5ae3 (0x930 bytes) 000327[0] 1761224326.730137 mapwrite 0x97b220 thru 0x97c634 (0x1415 bytes) 000333[0] 1761224326.846955 mapwrite 0x71738e thru 0x7184b2 (0x1125 bytes) 000334[0] 1761224326.899846 write 0x9f90fb thru 0x9f9bb6 (0xabc bytes) 000335[0] 1761224326.900745 trunc from 0x9f9bb7 to 0x61c233 (0x3dd983 bytes) 000337[0] 1761224326.955880 mapwrite 0x54e0a3 thru 0x54eb89 (0xae7 bytes) 000340[0] 1761224327.030297 read 0x0c146e thru 0x0c1ccd (0x860 bytes) 000341[0] 1761224327.057959 mapread 0x3d8952 thru 0x3da539 (0x1be8 bytes) 000343[1] 1761224327.074966 mapwrite 0x947824 thru 0x94877b (0xf58 bytes) 000346[0] 1761224327.214195 write 0x2ae993 thru 0x2b0253 (0x18c1 bytes) 000347[0] 1761224327.245459 write 0x21eae9 thru 0x21eb98 (0xb0 bytes) 000348[1] 1761224327.266048 mapwrite 0x2bc53f thru 0x2bcecc (0x98e bytes) 000350[0] 1761224327.335048 read 0x765505 thru 0x766d11 (0x180d bytes) 000353[0] 1761224327.374268 write 0x65cfa6 thru 0x65dcd3 (0xd2e bytes) 000354[1] 1761224327.410720 read 0x63305c thru 0x633e86 (0xe2b bytes) 000355[0] 1761224327.426671 write 0x21b0d1 thru 0x21bdd8 (0xd08 bytes) 000357[0] 1761224327.455712 mapread 0x592184 thru 0x59341f (0x129c bytes) 000358[1] 1761224327.482591 trunc from 0x94877c to 0x5320e6 (0x416695 bytes) 000359[0] 1761224327.526850 mapread 0x16598f thru 0x166c3b (0x12ad bytes) 000361[0] 1761224327.607329 read 0x485c3b thru 0x487901 (0x1cc7 bytes) 000362[1] 1761224327.635831 trunc from 0x5320e7 to 0x794554 (0x26246e bytes) 000365[1] 1761224327.757447 mapread 0x55b287 thru 0x55c80c (0x1586 bytes) 000366[1] 1761224327.771484 read 0x071f56 thru 0x073ca0 (0x1d4b bytes) 000370[0] 1761224327.869444 mapwrite 0x4f1fbf thru 0x4f39b2 (0x19f4 bytes) 000371[1] 1761224328.090407 mapwrite 0x0e02c6 thru 0x0e10c4 (0xdff bytes) 000376[0] 1761224328.224141 write 0x4d7d4a thru 0x4d9088 (0x133f bytes) 000379[1] 1761224328.284441 trunc from 0x794555 to 0x174d12 (0x61f842 bytes) truncating to largest ever: 0x99c3c2 000380[1] 1761224328.328402 trunc from 0x174d13 to 0x99c3c1 (0x8276af bytes) 000381[1] 1761224328.360094 trunc from 0x99c3c2 to 0x7c62d1 (0x1d60f0 bytes) 000382[1] 1761224328.378720 mapread 0x73c4e9 thru 0x73d2fa (0xe12 bytes) skipping zero size read 000386[1] 1761224328.466089 write 0x36c8a8 thru 0x36d328 (0xa81 bytes) 000387[1] 1761224328.506273 mapread 0x60a4e2 thru 0x60ad15 (0x834 bytes) 000389[0] 1761224328.551020 mapwrite 0x5ccccd thru 0x5cdc09 (0xf3d bytes) 000390[0] 1761224328.660857 read 0x4ffe6f thru 0x5012f9 (0x148b bytes) 000392[0] 1761224328.708077 mapread 0x5eadaf thru 0x5eaeed (0x13f bytes) 000393[0] 1761224328.744969 mapwrite 0x867cdd thru 0x868b4d (0xe71 bytes) 000396[0] 1761224328.872341 write 0x1f8011 thru 0x1f8a74 (0xa64 bytes) 000399[0] 1761224328.896970 read 0x43c2c3 thru 0x43d565 (0x12a3 bytes) 000400[0] 1761224328.920178 write 0x06d4b7 thru 0x06edbb (0x1905 bytes) 000401[1] 1761224328.972013 mapwrite 0x510be7 thru 0x511509 (0x923 bytes) 000403[1] 1761224329.077905 mapread 0x6eaa91 thru 0x6eb532 (0xaa2 bytes) 000406[0] 1761224329.139954 write 0x92d320 thru 0x92d7f9 (0x4da bytes) 000407[0] 1761224329.160186 write 0x910f48 thru 0x912378 (0x1431 bytes) 000409[1] 1761224329.191582 write 0x252c69 thru 0x25308c (0x424 bytes) 000410[1] 1761224329.265307 mapread 0x194a22 thru 0x195198 (0x777 bytes) 000411[0] 1761224329.275790 mapwrite 0x663dcb thru 0x665668 (0x189e bytes) 000413[0] 1761224329.453174 mapread 0x13a9bc thru 0x13b2e7 (0x92c bytes) 000414[0] 1761224329.518912 mapwrite 0x7f3b88 thru 0x7f3fb5 (0x42e bytes) 000415[0] 1761224329.606427 trunc from 0x92d7fa to 0x2e8534 (0x6452c5 bytes) 000417[0] 1761224329.700289 mapread 0x0848c1 thru 0x0865ec (0x1d2c bytes) 000420[0] 1761224329.797385 write 0x8c621e thru 0x8c70f6 (0xed9 bytes) 000422[1] 1761224329.799873 mapwrite 0x7b2557 thru 0x7b3b69 (0x1613 bytes) 000424[0] 1761224329.908845 write 0x5eb152 thru 0x5ebc99 (0xb48 bytes) 000428[1] 1761224329.936844 read 0x773e55 thru 0x774583 (0x72f bytes) 000431[0] 1761224329.971192 mapread 0x6bb6c0 thru 0x6bcc5a (0x159b bytes) 000432[1] 1761224329.984437 trunc from 0x8c70f7 to 0x04d523 (0x879bd3 bytes) 000435[1] 1761224330.064040 write 0x444808 thru 0x4450c4 (0x8bd bytes) 000436[0] 1761224330.066881 mapwrite 0x60adec thru 0x60b2e5 (0x4fa bytes) 000437[0] 1761224330.183971 mapwrite 0x26fdc2 thru 0x271b21 (0x1d60 bytes) 000438[1] 1761224330.281941 mapwrite 0x4533e9 thru 0x455236 (0x1e4e bytes) 000442[1] 1761224330.483571 read 0x04eb51 thru 0x04f206 (0x6b6 bytes) 000445[1] 1761224330.540785 read 0x2d6bdd thru 0x2d70ad (0x4d1 bytes) 000447[0] 1761224330.558435 mapwrite 0x65691a thru 0x657f49 (0x1630 bytes) 000448[1] 1761224330.656429 mapwrite 0x9a3d79 thru 0x9a52d1 (0x1559 bytes) 000450[0] 1761224330.811517 trunc from 0x9a52d2 to 0x77a197 (0x22b13a bytes) 000451[0] 1761224330.864585 trunc from 0x77a198 to 0x77069a (0x9afd bytes) 000453[0] 1761224330.924402 trunc from 0x77069b to 0x8a23e5 (0x131d4b bytes) 000454[1] 1761224330.993656 write 0x02be94 thru 0x02c237 (0x3a4 bytes) 000458[0] 1761224331.165357 trunc from 0x8a23e6 to 0x4e39b4 (0x3bea31 bytes) 000459[0] 1761224331.219354 read 0x24b938 thru 0x24d189 (0x1852 bytes) 000462[1] 1761224331.290774 trunc from 0x4e39b5 to 0x2a02f4 (0x2436c0 bytes) 000464[1] 1761224331.360696 write 0x6760c1 thru 0x677aed (0x1a2d bytes) 000465[1] 1761224331.388981 mapread 0x379f78 thru 0x37b353 (0x13dc bytes) 000466[0] 1761224331.425455 mapwrite 0x350f1f thru 0x352167 (0x1249 bytes) 000467[0] 1761224331.546764 write 0x2c6694 thru 0x2c7443 (0xdb0 bytes) 000470[1] 1761224331.603243 mapread 0x459144 thru 0x45a1b6 (0x1073 bytes) 000471[1] 1761224331.626520 read 0x208904 thru 0x208cee (0x3eb bytes) 000472[1] 1761224331.685280 mapwrite 0x5c7409 thru 0x5c7597 (0x18f bytes) truncating to largest ever: 0x9d440a 000474[0] 1761224331.788628 trunc from 0x677aee to 0x9d4409 (0x35c91c bytes) 000475[1] 1761224331.839688 trunc from 0x9d440a to 0x913fa7 (0xc0462 bytes) 000476[0] 1761224331.914822 mapwrite 0x277f37 thru 0x27896b (0xa35 bytes) 000477[0] 1761224332.076937 trunc from 0x913fa8 to 0x70acc1 (0x2092e6 bytes) 000480[0] 1761224332.150149 trunc from 0x70acc2 to 0x15d59d (0x5ad724 bytes) 000482[0] 1761224332.201929 trunc from 0x15d59e to 0x273d54 (0x1167b7 bytes) 000485[0] 1761224332.285828 read 0x1e4a6f thru 0x1e4b57 (0xe9 bytes) 000490[1] 1761224332.338998 mapread 0x098b47 thru 0x09a6f2 (0x1bac bytes) 000491[1] 1761224332.394354 read 0x1f7c76 thru 0x1f7cd8 (0x63 bytes) 000493[0] 1761224332.486039 read 0x11d63f thru 0x11da1d (0x3df bytes) 000495[1] 1761224332.545941 write 0x3e73ce thru 0x3e81c0 (0xdf3 bytes) All operations completed A-OK! PASS 16b (24s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 08:58:57 (1761224337) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (4s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 08:59:01 (1761224341) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.325311 s, 25.8 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.319284 s, 26.3 MB/s ldlm.namespaces.MGC192.168.201.138@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff9c32c91b1800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff9c32d0df3800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff9c32c91b1800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff9c32d0df3800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff9c32d0df3800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.51653 s, 12.3 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.37701 s, 44.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.06076 s, 50.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.27029 s, 19.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.81933 s, 15.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 3.69459 s, 28.4 MB/s PASS 16d (41s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 08:59:42 (1761224382) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.512505 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224351 s, 4.7 MB/s PASS 16e (8s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 08:59:50 (1761224390) TIMEOUT PASS 16f (32s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 09:00:23 (1761224423) TIMEOUT PASS 16g (27s) == sanityn test 16h: mmap read after truncate file ======= 09:00:50 (1761224450) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16h (7s) == sanityn test 16i: read after truncate file ============ 09:00:57 (1761224457) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (6s) == sanityn test 16j: race dio with buffered i/o ========== 09:01:04 (1761224464) 1+0 records in 1+0 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.42356 s, 18.4 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.987782 s, 26.5 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.15863 s, 12.1 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.35288 s, 19.4 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.87824 s, 14.0 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.08327 s, 12.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.57347 s, 10.2 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.960038 s, 27.3 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 1.35031 s, 19.4 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.93611 s, 8.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 3.04868 s, 8.6 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.1483 s, 12.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.27345 s, 11.5 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.38174 s, 11.0 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.58912 s, 10.1 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 2.92934 s, 8.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 3.13845 s, 8.4 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 3.28215 s, 8.0 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 3.469 s, 7.6 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (33s) == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 09:01:36 (1761224496) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: skipping zero size read test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported truncating to largest ever: 0x1a7062c fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x1a7062c skipping zero size read truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 Thu Oct 23 09:02:20 EDT 2025 + fsx 2 done: rc=0 Thu Oct 23 09:02:20 EDT 2025 + fsx 3 done: rc=0 Thu Oct 23 09:02:20 EDT 2025 + fsx 4 done: rc=0 Thu Oct 23 09:02:20 EDT 2025 + fsx 5 done: rc=0 Thu Oct 23 09:02:20 EDT 2025 PASS 16k (53s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 09:02:30 (1761224550) fail_loc=0x8000030a PASS 17 (10s) == sanityn test 18: mmap sanity check =========================================================================================== 09:02:40 (1761224560) mmap test1: basic mmap operation (PASS, 0.1638s) mmap test2: MAP_PRIVATE not write back (PASS, 0.11018s) mmap test3: concurrent mmap ops on two nodes (PASS, 12.527s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 4.3928s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.12167s) mmap test6: check mmap write/read content on two nodes (PASS, 0.21996s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: multi-thread mmap access (PASS, 1.31s) mmap test11: multi-thread shared mmap (PASS, 1.0501s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0728s) PASS 18 (29s) == sanityn test 19: test concurrent uncached read races ========================================================================= 09:03:09 (1761224589) oleg138-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg138-client: oleg138-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (4s) == sanityn test 20: test extra readahead page left in cache ============================================================== 09:03:13 (1761224593) PASS 20 (6s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 09:03:19 (1761224599) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (5s) == sanityn test 23: others should see updated atime while another read============================================================== 09:03:25 (1761224605) now is 1761224607 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7558 new atime is 1761224668 PASS 23 (68s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 09:04:33 (1761224673) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 388 490.2K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 399 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 114.5K 401 114.1K 1% /mnt/lustre[OST:1] filesystem_summary: 229.3K 388 229.0K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 388 490.2K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 399 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 401 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.3K 388 229.0K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 8.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 31.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 39.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502379 388 501991 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117999 399 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117265 401 116864 1% /mnt/lustre2[OST:1] filesystem_summary: 234852 388 234464 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 388 490.2K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 399 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 401 114.1K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.3K 388 229.0K 1% /mnt/lustre2 Reader error: 'No devices found' at 0 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502379 388 501991 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 117999 399 117600 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 117265 401 116864 1% /mnt/lustre[OST:1] filesystem_summary: 234852 388 234464 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502379 388 501991 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117999 399 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117265 401 116864 1% /mnt/lustre2[OST:1] filesystem_summary: 234852 388 234464 1% /mnt/lustre2 Reader error: 'No devices found' at 0 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 31744 3737600 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 39936 7498752 1% /mnt/lustre2 PASS 24a (6s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 09:04:39 (1761224679) PASS 24b (5s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 09:04:44 (1761224684) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (6s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 09:04:51 (1761224691) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (2s) == sanityn test 26a: allow mtime to get older ============ 09:04:54 (1761224694) utime: good mknod times 1761224694 <= 1761224695 <= 1761224695 for /mnt/lustre/f26a utime: good mknod times 1761224694 <= 1761224695 <= 1761224695 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (7s) == sanityn test 26b: sync mtime between ost and mds ====== 09:05:00 (1761224700) PASS 26b (7s) == sanityn test 26c: set-in-past on open file is not lost on close ========================================================== 09:05:08 (1761224708) multiop /mnt/lustre/f26c.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7558 File: /mnt/lustre/f26c.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2025-10-23 09:05:09.000000000 -0400 Birth: 2025-10-23 09:05:08.000000000 -0400 File: /mnt/lustre2/f26c.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2025-10-23 09:05:09.000000000 -0400 Birth: 2025-10-23 09:05:08.000000000 -0400 PASS 26c (7s) SKIP: sanityn test_27 skipping excluded test 27 == sanityn test 30: recreate file race =================== 09:05:16 (1761224716) 0 PASS 30 (8s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 09:05:24 (1761224724) fail_loc=0x314 PASS 31a (6s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 09:05:30 (1761224730) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg138-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.201.138@tcp:/lustre 7542784 40960 7497728 1% /mnt/lustre2 PASS 31b (19s) == sanityn test 31r: open-rename(replace) race =========== 09:05:49 (1761224749) -rw-r--r-- 1 root root 0 Oct 23 09:05 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (9s) == sanityn test 31s: open should not revalidate invalid dentry ========================================================== 09:05:58 (1761224758) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 1207: 42911 Terminated tail -f $tfile PASS 31s (7s) == sanityn test 31t: getattr should not revalidate invalid dentry ========================================================== 09:06:05 (1761224765) PASS 31t (7s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 09:06:14 (1761224774) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (3s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 09:06:17 (1761224777) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (3s) == sanityn test 33d: dependent transactions should trigger COS ========================================================== 09:06:20 (1761224780) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (3s) == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 09:06:23 (1761224783) SKIP: sanityn test_33e needs >= 2 MDTs SKIP 33e (3s) == sanityn test 34: no lock timeout under IO ============= 09:06:26 (1761224786) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg138-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9c32c91b1800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff9c32c91b1800.ost_server_uuid in IDLE state after 0 sec oleg138-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9c32c91b1800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff9c32c91b1800.ost_server_uuid in IDLE state after 0 sec oleg138-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9c32c91b1800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff9c32c91b1800.ost_server_uuid in IDLE state after 0 sec oleg138-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9c32c91b1800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff9c32c91b1800.ost_server_uuid in IDLE state after 0 sec oleg138-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9c32c91b1800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff9c32c91b1800.ost_server_uuid in IDLE state after 0 sec oleg138-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9c32c91b1800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff9c32c91b1800.ost_server_uuid in IDLE state after 0 sec PASS 34 (120s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 09:08:26 (1761224906) fail_loc=0x80000317 Race attempt 0 Wait for 49044 49125 for 60 sec... - open/close 757 (time 1761224922.12 total 10.01 last 75.61) - open/close 1618 (time 1761224932.13 total 20.02 last 86.05) - open/close 2471 (time 1761224942.14 total 30.03 last 85.18) - open/close 3300 (time 1761224952.14 total 40.04 last 82.87) total: 4000 open/close in 48.64 seconds: 82.24 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.201.138@tcp:/lustre 7.2G 41M 7.2G 1% /mnt/lustre 192.168.201.138@tcp:/lustre 7.2G 41M 7.2G 1% /mnt/lustre2 PASS 35 (71s) == sanityn test 36: handle ESTALE/open-unlink correctly == 09:09:37 (1761224977) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg138-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.07964 s, 48.6 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7558 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7496704) after_dd(7441408) after(7492608) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.00775 s, 52.0 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7558 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7492608) after_dd(7441408) after(7492608) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.11606 s, 47.0 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7558 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7492608) after_dd(7441408) after(7492608) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.961802 s, 54.5 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7558 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7492608) after_dd(7436288) after(7487488) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.944356 s, 55.5 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7558 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7487488) after_dd(7441408) after(7492608) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.06928 s, 49.0 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7558 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7492608) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.970553 s, 54.0 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7558 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7494656) after_dd(7439360) after(7490560) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.924063 s, 56.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7558 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7490560) after_dd(7441408) after(7492608) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.916177 s, 57.2 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7558 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7492608) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.817786 s, 64.1 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7558 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7494656) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.813369 s, 64.5 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7558 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7494656) after_dd(7443456) after(7494656) PASS 36 (272s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 09:14:09 (1761225249) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7558 - create 1556 (time 1761225262.88 total 10.00 last 155.53) - create 3057 (time 1761225272.88 total 20.01 last 150.05) - create 4580 (time 1761225282.89 total 30.01 last 152.29) - create 5975 (time 1761225292.89 total 40.01 last 139.48) - create 7513 (time 1761225302.89 total 50.01 last 153.73) - create 9055 (time 1761225312.89 total 60.02 last 154.15) total: 10000 create in 66.30 seconds: 150.83 ops/second PASS 37 (75s) == sanityn test 39a: file mtime does not change after rename ========================================================== 09:15:25 (1761225325) repeat after cancel_lru_locks PASS 39a (6s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 09:15:31 (1761225331) repeat after cancel_lru_locks PASS 39b (7s) == sanityn test 39c: check truncate mtime update ================================================================================ 09:15:38 (1761225338) repeat after cancel_lru_locks PASS 39c (6s) == sanityn test 39d: sync write should update mtime ====== 09:15:45 (1761225345) fail_loc=0x411 fail_loc=0 PASS 39d (6s) SKIP: sanityn test_40a skipping ALWAYS excluded test 40a == sanityn test 40b: pdirops: open|create and others ======================================================================== 09:15:51 (1761225351) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (15s) == sanityn test 40c: pdirops: link and others ======================================================================== 09:16:06 (1761225366) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (16s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 09:16:22 (1761225382) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (15s) == sanityn test 40e: pdirops: rename and others ======================================================================== 09:16:37 (1761225397) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (14s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 09:16:51 (1761225411) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (12s) == sanityn test 41b: pdirops: create vs create ======================================================================== 09:17:03 (1761225423) open: File exists Conflict PASS 41b (10s) == sanityn test 41c: pdirops: create vs link ======================================================================== 09:17:14 (1761225434) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (11s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 09:17:25 (1761225445) Conflict PASS 41d (12s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 09:17:37 (1761225457) Conflict PASS 41e (14s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 09:17:50 (1761225470) Conflict PASS 41f (12s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 09:18:02 (1761225482) Conflict PASS 41g (12s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 09:18:15 (1761225495) Conflict PASS 41h (12s) == sanityn test 41i: reint_open: create vs create ======== 09:18:27 (1761225507) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (960s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 09:34:27 (1761226467) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (10s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 09:34:37 (1761226477) open: File exists Conflict PASS 42b (12s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 09:34:49 (1761226489) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (14s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 09:35:03 (1761226503) Conflict PASS 42d (12s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 09:35:15 (1761226515) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (13s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 09:35:28 (1761226528) Conflict PASS 42f (11s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 09:35:39 (1761226539) Conflict PASS 42g (11s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 09:35:50 (1761226550) Conflict PASS 42h (12s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 09:36:02 (1761226562) PASS 43a (62s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 09:37:04 (1761226624) Conflict PASS 43b (11s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 09:37:15 (1761226635) Conflict PASS 43c (12s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 09:37:27 (1761226647) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (13s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 09:37:41 (1761226661) Conflict PASS 43e (12s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 09:37:52 (1761226672) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (12s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 09:38:04 (1761226684) stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (12s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 09:38:16 (1761226696) Conflict PASS 43h (13s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 09:38:29 (1761226709) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (4s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 09:38:33 (1761226713) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (122s) == sanityn test 43k: unlink vs create ==================== 09:40:35 (1761226835) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (1042s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 09:57:57 (1761227877) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Thu Oct 23 09:58:04 EDT 2025 PASS 44a (12s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 09:58:09 (1761227889) open: File exists Conflict PASS 44b (12s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 09:58:21 (1761227901) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (12s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 09:58:34 (1761227914) Conflict PASS 44d (12s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 09:58:46 (1761227926) Conflict PASS 44e (13s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 09:58:59 (1761227939) Conflict PASS 44f (12s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 09:59:11 (1761227951) Conflict PASS 44g (11s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 09:59:22 (1761227962) Conflict PASS 44h (11s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 09:59:34 (1761227974) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (2s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 09:59:36 (1761227976) PASS 45a (105s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 10:01:21 (1761228081) Conflict PASS 45b (9s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 10:01:30 (1761228090) Conflict PASS 45c (11s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 10:01:41 (1761228101) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (12s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 10:01:53 (1761228113) Conflict PASS 45e (11s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 10:02:04 (1761228124) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (12s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 10:02:16 (1761228136) stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (11s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 10:02:27 (1761228147) Conflict PASS 45h (10s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 10:02:37 (1761228157) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (3s) == sanityn test 45j: read vs rename ====================== 10:02:40 (1761228160) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (952s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 10:18:32 (1761229112) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (12s) == sanityn test 46b: pdirops: link vs create ======================================================================== 10:18:44 (1761229124) open: File exists Conflict PASS 46b (11s) == sanityn test 46c: pdirops: link vs link ======================================================================== 10:18:55 (1761229135) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (10s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 10:19:05 (1761229145) Conflict PASS 46d (10s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 10:19:15 (1761229155) Conflict PASS 46e (10s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 10:19:25 (1761229165) Conflict PASS 46f (10s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 10:19:35 (1761229175) Conflict PASS 46g (12s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 10:19:47 (1761229187) Conflict PASS 46h (10s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 10:19:58 (1761229198) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (2s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 10:20:00 (1761229200) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (2s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 10:20:02 (1761229202) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (2s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 10:20:04 (1761229204) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (2s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 10:20:06 (1761229206) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (2s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 10:20:08 (1761229208) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (2s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 10:20:10 (1761229210) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (2s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 10:20:12 (1761229212) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (3s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 10:20:15 (1761229215) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.097729 s, 105 kB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (12s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 10:20:27 (1761229227) 0+1 records in 0+1 records out 158 bytes copied, 0.0330849 s, 4.8 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3510: kill: (259991) - No such process PASS 51a (9s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 10:20:36 (1761229236) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0158542 s, 64.6 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0655336 s, 16.0 MB/s 3145728 PASS 51b (20s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 10:20:56 (1761229256) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0274885 s, 37.3 kB/s PASS 51c (9s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 10:21:05 (1761229265) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327295 s, 32.0 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (6s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 10:21:12 (1761229272) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2450 0x992 0x240000400 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2450 0x992 0x240000400 read lease(1) has applied. PASS 51e (8s) == sanityn test 54: rename locking ======================= 10:21:19 (1761229279) llite.lustre-ffff9c32c91b1800.xattr_cache=1 llite.lustre-ffff9c32d0df3800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505011 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-23 10:21:20.000000000 -0400 Modify: 2025-10-23 10:21:20.000000000 -0400 Change: 2025-10-23 10:21:20.000000000 -0400 Birth: 2025-10-23 10:21:19.000000000 -0400 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505014 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-23 10:21:26.000000000 -0400 Modify: 2025-10-23 10:21:27.000000000 -0400 Change: 2025-10-23 10:21:27.000000000 -0400 Birth: 2025-10-23 10:21:25.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505017 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-23 10:21:33.000000000 -0400 Modify: 2025-10-23 10:21:33.000000000 -0400 Change: 2025-10-23 10:21:33.000000000 -0400 Birth: 2025-10-23 10:21:31.000000000 -0400 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505016 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-23 10:21:33.000000000 -0400 Modify: 2025-10-23 10:21:34.000000000 -0400 Change: 2025-10-23 10:21:34.000000000 -0400 Birth: 2025-10-23 10:21:31.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505020 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-23 10:21:39.000000000 -0400 Modify: 2025-10-23 10:21:40.000000000 -0400 Change: 2025-10-23 10:21:40.000000000 -0400 Birth: 2025-10-23 10:21:38.000000000 -0400 File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505019 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-10-23 10:21:39.000000000 -0400 Modify: 2025-10-23 10:21:40.000000000 -0400 Change: 2025-10-23 10:21:40.000000000 -0400 Birth: 2025-10-23 10:21:38.000000000 -0400 PASS 54 (31s) == sanityn test 55a: rename vs unlink target dir ========= 10:21:51 (1761229311) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (11s) == sanityn test 55b: rename vs unlink source dir ========= 10:22:03 (1761229323) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (13s) == sanityn test 55c: rename vs unlink orphan target dir == 10:22:15 (1761229335) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (19s) == sanityn test 55d: rename file vs link ================= 10:22:34 (1761229354) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (16s) == sanityn test 55e: rename race AB/BA under the same parent dir ========================================================== 10:22:50 (1761229370) SKIP: sanityn test_55e needs >= 2 MDTs SKIP 55e (3s) == sanityn test 55f: rename: (P1/A -> P2/B) race with (P2/B -> P1/A) ========================================================== 10:22:53 (1761229373) fail_loc=0x155 We survived after P1A_P2B/P2B_P1A race test fail_loc=0x0 PASS 55f (18s) == sanityn test 55g: rename: race with trylock =========== 10:23:11 (1761229391) mdt.lustre-MDT0000.enable_rename_trylock=1 fail_loc=0x155 mv: cannot move '/mnt/lustre/d55g.sanityn/a1/a2' to a subdirectory of itself, '/mnt/lustre/d55g.sanityn/b1/b2/b3/b4/a2' We survived race with trylock test fail_loc=0x0 PASS 55g (22s) == sanityn test 56a: test llverdev with single large stripe ========================================================== 10:23:33 (1761229413) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1761229414 chunksize: 4096 size: 16777216000000 write offset: 16384000000kB 0.726656 MB/s read offset: 16384000000kB inf MB/s llverdev: data verified successfully PASS 56a (90s) == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 10:25:03 (1761229503) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1761229507 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 1.32689 MB/s read offset: 32768000000kB inf MB/s llverdev: data verified successfully PASS 56b (105s) == sanityn test 60: Verify data_version behaviour ======== 10:26:48 (1761229608) PASS 60 (11s) == sanityn test 70a: cd directory && rm directory ======== 10:26:59 (1761229619) PASS 70a (8s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 10:27:07 (1761229627) PASS 70b (9s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 10:27:16 (1761229636) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0147442 s, 4.4 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (3s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 10:27:20 (1761229640) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0123338 s, 5.3 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (5s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 10:27:24 (1761229644) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (3s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 10:27:28 (1761229648) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0440392 s, 1.5 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (4s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 10:27:32 (1761229652) llite.lustre-ffff9c32c91b1800.xattr_cache=1 llite.lustre-ffff9c32d0df3800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (9s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 10:27:41 (1761229661) llite.lustre-ffff9c32c91b1800.xattr_cache=1 llite.lustre-ffff9c32d0df3800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (9s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 10:27:50 (1761229670) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0515942 s, 19.8 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0240084 s, 42.7 kB/s 272804: get lock2 272806: get lock1 272806: done 272806: get lock2 272804: done 272804: get lock1 fcntl cmd 7 failed: Resource deadlock avoided 272806: cannot get lock2: Resource deadlock avoided 272806: exit rc=0 fcntl cmd 7 failed: Resource deadlock avoided 272804: cannot get lock1: Resource deadlock avoided 272804: done 272804: exit rc=0 PASS 74 (12s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 10:28:03 (1761229683) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0875015 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.045367 s, 23.1 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4193: 273388 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (13s) == sanityn test 76: Verify MDT open_files listing ======== 10:28:16 (1761229696) opening files: ........ closing files: ........ PASS 76 (78s) == sanityn test 77a: check FIFO NRS policy =============== 10:29:34 (1761229774) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.377168 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0662267 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138993 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127638 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230932 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182667 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.300551 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.291323 s, 3.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.270363 s, 3.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248079 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.266529 s, 3.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.386686 s, 2.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.320286 s, 3.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.341628 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.304177 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.344908 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.407435 s, 2.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9c32c91b1800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100911 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0686154 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033963 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127044 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144985 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232783 s, 4.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194828 s, 5.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.172721 s, 6.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731631 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.343912 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11774 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105491 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0827716 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101092 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0730727 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.06457 s, 16.2 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (11s) == sanityn test 77b: check CRR-N NRS policy ============== 10:29:46 (1761229786) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.635996 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0623372 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131857 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120627 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166047 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120701 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182035 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.093869 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223824 s, 4.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166212 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.32065 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.285552 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.385854 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.375172 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.498039 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.525565 s, 2.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.593822 s, 1.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9c32c91b1800.lock_unused_count=6 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179152 s, 5.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0973275 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00452958 s, 231 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112296 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115591 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185254 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203499 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140773 s, 74.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155053 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123006 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113653 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119137 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128855 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120386 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0918731 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.252645 s, 4.2 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.620504 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0600421 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0721124 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107651 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165715 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177842 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131665 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.326198 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152864 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.234595 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205489 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.238296 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.450412 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.328125 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.416479 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.505108 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0741303 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0027731 s, 378 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742479 s, 141 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9c32c91b1800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.435539 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00385756 s, 272 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0040749 s, 257 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135445 s, 77.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660201 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208559 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.238938 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.262047 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018232 s, 57.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158946 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201994 s, 51.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203922 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166091 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0836317 s, 12.5 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (15s) == sanityn test 77c: check ORR NRS policy ================ 10:30:01 (1761229801) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.551334 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274359 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0622234 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313016 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.072605 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101602 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0779683 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114136 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155282 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103233 s, 10.2 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22103 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232243 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.26255 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251321 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.290172 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166427 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.326446 s, 3.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00467864 s, 224 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0510227 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0994063 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00298609 s, 351 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0561189 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760625 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0912492 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0626336 s, 16.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0607644 s, 17.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0715182 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11232 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0684797 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149697 s, 70.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964282 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0393561 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0833209 s, 12.6 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg138-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg138-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.503426 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0679243 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0778212 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163487 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198686 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209056 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222917 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22779 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228641 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.231919 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.294375 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.326587 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.296737 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.313568 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.313193 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.378903 s, 2.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9c32c91b1800.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.459198 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00306657 s, 342 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00272824 s, 384 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00245086 s, 428 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185718 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253687 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.293424 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.234392 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196941 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.287963 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.346602 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150943 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118455 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108379 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104285 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0857727 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0825484 s, 12.7 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.504316 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0634232 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0871541 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218005 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189911 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217083 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.254211 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.266751 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248581 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.286825 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.306537 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.292202 s, 3.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.315324 s, 3.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.265505 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.319873 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.312613 s, 3.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9c32c91b1800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.307918 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0995887 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00422066 s, 248 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0691354 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0978721 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0812503 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00434308 s, 241 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111278 s, 9.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0732589 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0541053 s, 19.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0509138 s, 20.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313412 s, 33.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273378 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216684 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0490605 s, 21.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0400301 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189341 s, 55.4 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg138-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg138-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (18s) == sanityn test 77d: check TRR nrs policy ================ 10:30:18 (1761229818) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.460444 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0518889 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129917 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194299 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197258 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181194 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18153 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.289598 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.280066 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253526 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.259544 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.306193 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.267489 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.355202 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.398879 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.378632 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.462202 s, 2.3 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00264369 s, 397 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889762 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.37591 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178212 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.37485 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00330143 s, 318 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.268422 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.21894 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.37639 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16312 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180249 s, 5.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154934 s, 6.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115006 s, 9.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0907898 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0507725 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0906619 s, 11.6 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg138-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg138-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.498447 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0643013 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0709677 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0507901 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112773 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129609 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223354 s, 4.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173821 s, 6.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.210901 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119059 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193445 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.281948 s, 3.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9c32c91b1800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.254978 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.366647 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.356403 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.357226 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.392189 s, 2.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0624157 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151978 s, 69.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146153 s, 71.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179695 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.293075 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153991 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151015 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140195 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148492 s, 7.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202891 s, 51.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142746 s, 7.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107861 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0677307 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0448778 s, 23.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145134 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0597179 s, 17.6 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg138-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg138-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.491604 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0513327 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0610859 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0776054 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0679634 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203113 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188103 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218398 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.243008 s, 4.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.322672 s, 3.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.333291 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.339342 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.334637 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.335641 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.353997 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.283592 s, 3.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9c32c91b1800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.335487 s, 3.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152044 s, 69.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153058 s, 68.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00378281 s, 277 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163475 s, 6.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192195 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180261 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616217 s, 170 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147297 s, 7.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.243173 s, 4.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.221139 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168424 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0943 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0575788 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0667428 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315194 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0406491 s, 25.8 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (20s) == sanityn test 77e: check TBF NID nrs policy ============ 10:30:38 (1761229838) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.201.38@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.678774 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0491564 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126422 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106776 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139759 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127988 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187648 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.236521 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.305049 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.353094 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.352423 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.356253 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.358686 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.384959 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.463776 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.465503 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.523545 s, 2.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9c32c91b1800.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0977246 s, 10.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0510064 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140635 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0720044 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0716861 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0729703 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0765311 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0741126 s, 14.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0694792 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0783452 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0924778 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0905451 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103483 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103483 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111179 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129362 s, 8.1 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.647448 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0928059 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110289 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123149 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101617 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0833441 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241224 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.256459 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.25934 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.372431 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.366142 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.282446 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.340138 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131221 s, 8.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.363511 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157102 s, 66.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.448351 s, 2.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794117 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252787 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134909 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143267 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.419201 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155371 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737588 s, 142 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755055 s, 139 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00599815 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.092055 s, 11.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227462 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0911034 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193715 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0561369 s, 18.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9c32c91b1800.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.593377 s, 1.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0649285 s, 16.1 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.630051 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0832162 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0553172 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0885147 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200175 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101986 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181526 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143915 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213222 s, 4.9 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187638 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.324392 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.292158 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.204907 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.352638 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.320816 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.31308 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.438652 s, 2.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9c32c91b1800.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129995 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0029643 s, 354 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568987 s, 184 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00517685 s, 203 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0852011 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148319 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128575 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0684852 s, 15.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.04207 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0359215 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0707792 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029595 s, 35.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129516 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0878775 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202402 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654372 s, 160 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.424118 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0969443 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0822622 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0530437 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129886 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.303421 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232406 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.234191 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.300099 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.372396 s, 2.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9c32c91b1800.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.305761 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.333958 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.410506 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.35973 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.363069 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.311558 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.38478 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121708 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122033 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110681 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132828 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14877 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181043 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129115 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149781 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150048 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155099 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134671 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139424 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1414 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144739 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148169 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141029 s, 7.4 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (28s) == sanityn test 77f: check TBF JobID nrs policy ========== 10:31:06 (1761229866) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.560859 s, 29.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0653428 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.084469 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131953 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193127 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.214927 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.252913 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.265178 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.285934 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.318217 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.323916 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.344739 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.352429 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.353873 s, 3.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.359131 s, 2.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.373149 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.432653 s, 2.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9c32c91b1800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=3 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00303086 s, 346 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00505694 s, 207 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114665 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00291969 s, 359 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014815 s, 70.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125682 s, 8.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159062 s, 6.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115571 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178682 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0871637 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010261 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0555792 s, 18.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0382681 s, 27.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0481618 s, 21.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0748244 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0793156 s, 13.2 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.558719 s, 30.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107075 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0704426 s, 14.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18725 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.246067 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.313851 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.399484 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.353517 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.321617 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.415878 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.371555 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.455931 s, 2.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.318739 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.50873 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.319996 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.497416 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.386967 s, 2.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00476673 s, 220 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145493 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00384787 s, 273 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729621 s, 144 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330004 s, 31.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069137 s, 152 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767086 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0637242 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568577 s, 184 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121964 s, 8.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122851 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0520096 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.09347 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0587565 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0614945 s, 17.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141436 s, 7.4 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg138-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg138-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.493108 s, 34.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105302 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123652 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125669 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137572 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149473 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0503135 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0925572 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193651 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208083 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135939 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200305 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237047 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.332308 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.318617 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.34414 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.349546 s, 3.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0494571 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0510178 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0550571 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00593677 s, 177 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00249023 s, 421 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114935 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136206 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0890986 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10374 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124391 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00607903 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113989 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823091 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132905 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0393898 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0836039 s, 12.5 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg138-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg138-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.427124 s, 39.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860119 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1241 s, 8.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.278303 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.324256 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.405855 s, 2.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.402901 s, 2.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.383797 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.404424 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.385568 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.35117 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.44468 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.421973 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.427408 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.488705 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.437333 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.569527 s, 1.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00250561 s, 418 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00270264 s, 388 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00309763 s, 339 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00271048 s, 387 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010622 s, 98.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179312 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.271158 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00317614 s, 330 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217892 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199074 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175623 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.233403 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0974206 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0802798 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0627221 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0347205 s, 30.2 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (28s) == sanityn test 77g: Change TBF type directly ============ 10:31:34 (1761229894) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (12s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 10:31:46 (1761229906) oleg138-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg138-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg138-client: oleg138-server: ssh exited with exit code 2 oleg138-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Invalid argument oleg138-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Invalid argument pdsh@oleg138-client: oleg138-server: ssh exited with exit code 22 oleg138-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Invalid argument oleg138-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Invalid argument pdsh@oleg138-client: oleg138-server: ssh exited with exit code 22 oleg138-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Invalid argument oleg138-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Invalid argument pdsh@oleg138-client: oleg138-server: ssh exited with exit code 22 PASS 77h (14s) == sanityn test 77i: Change rank of TBF rule ============= 10:32:00 (1761229920) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (27s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 10:32:27 (1761229947) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.763253 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0639712 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0412645 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.091011 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194154 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180329 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224771 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201997 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244433 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217175 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.256881 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.28364 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.298078 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.304109 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.2987 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.291007 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.331069 s, 3.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9c32c91b1800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.226745 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.28007 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.279819 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.221569 s, 4.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237691 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23647 s, 4.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.247355 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.256348 s, 4.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253289 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.291496 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253598 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.262864 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255313 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.325408 s, 3.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.326854 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.259636 s, 4.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=4 removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.26149 s, 11.3 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6935 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (71s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 10:33:38 (1761230018) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg138-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg138-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg138-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg138-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.3154 s, 12.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0891716 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104043 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141441 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.204643 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189124 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185778 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187225 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228343 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227182 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22275 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229623 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.249116 s, 4.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244399 s, 4.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.271406 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.353773 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.355498 s, 2.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0837451 s, 12.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0723877 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0757384 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0777292 s, 13.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.098416 s, 10.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0892581 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.086214 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0985796 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110405 s, 9.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0978344 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111843 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126648 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126152 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130766 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12892 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119537 s, 8.8 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg138-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg138-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.1232 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9992 s, 5.2 MB/s Read runtime is 22 s, speed is 4.545454 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg138-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg138-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg138-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg138-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.29229 s, 13.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0921155 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137783 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180419 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160802 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154185 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237222 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.276976 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.297184 s, 3.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.370509 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.344926 s, 3.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.326823 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245745 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.3657 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.402525 s, 2.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.394023 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.432654 s, 2.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0424242 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0796423 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0640512 s, 16.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0494102 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0609794 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0686987 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0615947 s, 17.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0726323 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0759641 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0734386 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0770905 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0967017 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0908397 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0880602 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0906366 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0988487 s, 10.6 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg138-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg138-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.0956 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.0565 s, 5.2 MB/s Read runtime is 22 s, speed is 4.545454 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (155s) == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 10:36:13 (1761230173) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg138-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg138-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg138-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg138-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.34278 s, 12.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134595 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155748 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179251 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159296 s, 6.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147459 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213942 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.225143 s, 4.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.258425 s, 4.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253677 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255112 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.279152 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.268094 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.321221 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.324021 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.387781 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.480716 s, 2.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0906907 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0956291 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0955566 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0824602 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0794287 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0855086 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0855094 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.081377 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0799101 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0933554 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0982732 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115326 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0944245 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119004 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0836781 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112233 s, 9.3 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9844 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8887 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg138-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg138-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg138-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg138-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.29032 s, 13.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103879 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119628 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119749 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150197 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175697 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181809 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.21785 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165115 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.234733 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184843 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22544 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.233556 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237954 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260816 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.262599 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.328577 s, 3.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105273 s, 10.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0924945 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102188 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115699 s, 9.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117459 s, 8.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107518 s, 9.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159481 s, 6.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15609 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118754 s, 8.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122845 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152231 s, 6.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153475 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138825 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136993 s, 7.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154714 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151091 s, 6.9 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.0906 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.819 s, 5.3 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (158s) == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 10:38:51 (1761230331) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.876922 s, 19.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0394525 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.086114 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0551959 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0907665 s, 11.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0933158 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.212948 s, 4.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9c32c91b1800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162388 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224551 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.24384 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.26931 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.28558 s, 3.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198865 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.277252 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128527 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.314125 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.35456 s, 3.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied ldlm.namespaces.lustre-OST0000-osc-ffff9c32c91b1800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00436757 s, 240 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665647 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1439 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206374 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00257665 s, 407 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155617 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11597 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12475 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19609 s, 5.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0763624 s, 13.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0968614 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0651568 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0610306 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0516107 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163447 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0596687 s, 17.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.03938 s, 11.6 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.0051 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.38@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.38@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.837771 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0692274 s, 15.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109624 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0986349 s, 10.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0730351 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162513 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.25671 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.250935 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.360584 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.38952 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.415764 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.489904 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.466272 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.396042 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.487645 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.596332 s, 1.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.538817 s, 1.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9c32c91b1800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.312579 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.212214 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193773 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207924 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196937 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189557 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240314 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196822 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213853 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.246914 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.34681 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.250301 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220493 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224819 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.233344 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.247547 s, 4.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg138-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.59435 s, 12.2 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.96598 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.38@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.477908 s, 35.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0402579 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0599452 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0710762 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0624984 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0513624 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0752528 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217501 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.270316 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.290162 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.279119 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.286047 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.30007 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.316176 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.320104 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.326778 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.347208 s, 3.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00311971 s, 336 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00317214 s, 331 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00334278 s, 314 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00355834 s, 295 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00568337 s, 184 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771726 s, 136 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00261524 s, 401 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833223 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0668553 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0605192 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00328172 s, 320 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0925027 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0916166 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0403173 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0582044 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312905 s, 33.5 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg138-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.67101 s, 10.8 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.15924 s, 20.3 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.513733 s, 32.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.032685 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0758132 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0481582 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0750919 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0985746 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118731 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150278 s, 7.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148762 s, 7.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163392 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173096 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144385 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152381 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195784 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201112 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158525 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229637 s, 4.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011515 s, 91.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00385618 s, 272 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00499034 s, 210 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821255 s, 128 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133461 s, 78.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698596 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00424057 s, 247 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142725 s, 7.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0953565 s, 11.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108821 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0917237 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114273 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011936 s, 87.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118652 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0545841 s, 19.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163086 s, 6.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg138-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.7394 s, 9.8 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.94447 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.3761 s, 11.2 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.0056 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.377756 s, 44.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107073 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135008 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244509 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.250267 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.214641 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.332693 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.256454 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.354737 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.275511 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.326575 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.381919 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.336147 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.297125 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.425436 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.324441 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.285565 s, 3.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134493 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156234 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0912447 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00218483 s, 480 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125652 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16446 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165935 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103724 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174208 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163998 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157119 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174257 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.236756 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0461853 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0480593 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137359 s, 7.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg138-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.0962 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6346 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.542949 s, 30.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0354091 s, 29.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0398165 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.07739 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0921016 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.265579 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.287125 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273274 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.314327 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.339978 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.324591 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.337304 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.373543 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.358529 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.357516 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.373988 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.393938 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692583 s, 151 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602997 s, 174 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0367024 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178781 s, 58.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299731 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0539865 s, 19.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163708 s, 64.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710076 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136511 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.096984 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0669347 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00245096 s, 428 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0620946 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0500983 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152194 s, 68.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109067 s, 96.1 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 7.88688 s, 13.3 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.9143 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.448281 s, 37.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0506654 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0604974 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16668 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158701 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155024 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162782 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.219025 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183817 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228754 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.214027 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244296 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215075 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23676 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.272894 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.24681 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.256716 s, 4.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00228675 s, 459 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114043 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00215023 s, 488 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143331 s, 73.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00209422 s, 501 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00489816 s, 214 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182374 s, 57.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134524 s, 77.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0819012 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.074558 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00298571 s, 351 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0584078 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0644307 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0725335 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103616 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435379 s, 24.1 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.4815 s, 10.0 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.91396 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.6887 s, 9.8 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.95894 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (427s) == sanityn test 77kb: Check different granular TBF type combination: jobid+opcode ========================================================== 10:45:58 (1761230758) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.548145 s, 30.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0593035 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0576953 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.189521 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174509 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185495 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190061 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20422 s, 5.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.238001 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218428 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.262312 s, 4.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.21822 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.275147 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215981 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.280457 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.24381 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.276786 s, 3.8 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=2 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0773885 s, 13.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00564786 s, 186 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321553 s, 32.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0342537 s, 30.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020334 s, 51.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00283885 s, 369 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0956544 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00483454 s, 217 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0882959 s, 11.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110007 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00285284 s, 368 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140691 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028476 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0325644 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0401315 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021287 s, 49.3 MB/s removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77kb.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.84131 s, 15.3 MB/s Write runtime is 8 s, speed is 12.500000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.90115 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kb (48s) == sanityn test 77kc: Check different granular TBF type combination: nid+opcode ========================================================== 10:46:46 (1761230806) ost.OSS.ost_io.nrs_policies=tbf nid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.38@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.38@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.414101 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271794 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0813788 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0837843 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0976968 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127528 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109378 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132611 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.204068 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.21885 s, 4.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156601 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175106 s, 6.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.236192 s, 4.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188594 s, 5.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1685 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158694 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208927 s, 5.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0707887 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0745711 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0709908 s, 14.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0699049 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0857299 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0795495 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0386743 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0337396 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0863608 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0791378 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0805701 s, 13.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0652264 s, 16.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0422742 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.09773 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.053479 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0550825 s, 19.0 MB/s removed '/mnt/lustre/d77kc.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77kc.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77kc.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.98066 s, 15.0 MB/s Write runtime is 8 s, speed is 12.500000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.87487 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kc.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kc (45s) == sanityn test 77kd: Check different granular TBF type combination: nid+jobid ========================================================== 10:47:32 (1761230852) ost.OSS.ost_io.nrs_policies=tbf nid+jobid ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.38@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kd.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.427489 s, 39.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0581569 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0854225 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0800324 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170136 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186317 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228948 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.204413 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255043 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207681 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.275637 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.292551 s, 3.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.283211 s, 3.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255797 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.325803 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.319226 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.30768 s, 3.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0581468 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.052302 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269206 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0612562 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280233 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0554799 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0573523 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0359522 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324335 s, 32.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0480882 s, 21.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0637318 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0484791 s, 21.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0494227 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0410279 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0423694 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0647495 s, 16.2 MB/s removed '/mnt/lustre/d77kd.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77kd.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77kd.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kd.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.49987 s, 16.1 MB/s Write runtime is 8 s, speed is 12.500000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kd.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.0935 s, 20.6 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77kd.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kd (39s) == sanityn test 77ke: Check different granular TBF type combination: jobid+opcode ========================================================== 10:48:11 (1761230891) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ke.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.65834 s, 25.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0636558 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0463899 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0652886 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0723397 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183723 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133508 s, 7.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22944 s, 4.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.230897 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.266962 s, 3.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.237073 s, 4.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.262222 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.228424 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.349492 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.37533 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.389582 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.429459 s, 2.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0460478 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.035351 s, 29.7 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0721971 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.043371 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248019 s, 42.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0711377 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0470514 s, 22.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716525 s, 146 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110559 s, 94.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00255879 s, 410 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00303848 s, 345 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0489656 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301405 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022935 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018588 s, 56.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103497 s, 101 MB/s removed '/mnt/lustre/d77ke.sanityn/nrs_r_oleg138-client.virtnet' removed '/mnt/lustre/d77ke.sanityn/nrs_w_oleg138-client.virtnet' removed directory '/mnt/lustre/d77ke.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ke.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.2862 s, 10.2 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ke.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.98522 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77ke.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 7.17604 s, 14.6 MB/s Write runtime is 8 s, speed is 12.500000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.95888 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77ke.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77ke (89s) == sanityn test 77kf: Check different granular TBF type combination: uid+gid ========================================================== 10:49:40 (1761230980) ost.OSS.ost_io.nrs_policies=tbf uid+gid ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kf.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.980441 s, 17.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0603222 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0838829 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141783 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0991655 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.206105 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222872 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200251 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213038 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.235203 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201867 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.257535 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253735 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.301798 s, 3.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.333378 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.288498 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.310801 s, 3.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0492831 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0529247 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0912321 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647489 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128538 s, 8.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0760339 s, 13.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0846091 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0626742 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.049982 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0553144 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0415787 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0361202 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0432781 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841438 s, 125 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268466 s, 39.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0339534 s, 30.9 MB/s removed '/mnt/lustre/d77kf.sanityn/nrs_r_oleg138-client.virtnet' removed '/mnt/lustre/d77kf.sanityn/nrs_w_oleg138-client.virtnet' removed directory '/mnt/lustre/d77kf.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kf.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8471 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kf.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9188 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77kf.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kf (71s) == sanityn test 77kg: check different granular TBF type combination: uid+opcode ========================================================== 10:50:51 (1761231051) ost.OSS.ost_io.nrs_policies=tbf uid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.596342 s, 28.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.074048 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0714981 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160675 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19408 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19888 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.219385 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209637 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.225539 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.231819 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.227101 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23534 s, 4.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.254801 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.256678 s, 4.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251227 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.26704 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.298165 s, 3.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff9c32c91b1800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff9c32c91b1800.lock_unused_count=7 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214448 s, 48.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817287 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0414534 s, 25.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0354823 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0607962 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0714876 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125698 s, 8.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0503854 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026989 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0334294 s, 31.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0385308 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228991 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152442 s, 68.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0701693 s, 14.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0660714 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168777 s, 62.1 MB/s removed '/mnt/lustre/d77kg.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77kg.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77kg.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 6.58181 s, 15.9 MB/s Write runtime is 8 s, speed is 12.500000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.84712 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/nrs_r_oleg138-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.400669 s, 41.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0618455 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0494378 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122677 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187804 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185011 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178966 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195396 s, 5.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217623 s, 4.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194934 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.210265 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220629 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.250319 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.256782 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.306969 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.272634 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273577 s, 3.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0619946 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782708 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00309281 s, 339 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00585795 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676309 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0852727 s, 12.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0527143 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115209 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0370362 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860136 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0887917 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261846 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289281 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201871 s, 51.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0741977 s, 14.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100298 s, 10.5 MB/s removed '/mnt/lustre/d77kg.sanityn/nrs_w_oleg138-client.virtnet' removed '/mnt/lustre/d77kg.sanityn/nrs_r_oleg138-client.virtnet' removed directory '/mnt/lustre/d77kg.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.6213 s, 9.9 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.9228 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.1506 s, 10.3 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.8986 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kg (134s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 10:53:05 (1761231185) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (10s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 10:53:16 (1761231196) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg138-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.46081 s, 303 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg138-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync]