-----============= acceptance-small: sanityn ============----- Sun Jan 4 23:06:27 EST 2026 mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky oleg454-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 40a 102 skipping tests SLOW=no: 33a === sanityn: start setup 23:06:34 (1767585994) === Starting client oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 Mount client oleg454-client.virtnet: mount -t lustre -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 Started clients oleg454-client.virtnet: 192.168.204.154@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) oleg454-client.virtnet: executing check_config_client /mnt/lustre oleg454-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg454-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff993903017800.idle_timeout=debug osc.lustre-OST0000-osc-ffff9939106e5800.idle_timeout=debug osc.lustre-OST0001-osc-ffff993903017800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9939106e5800.idle_timeout=debug disable quota as required oleg454-server: oleg454-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 23:06:44 (1767586004) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018709 s, 56.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7553] == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 23:06:45 (1767586005) PASS 0 (3s) == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 23:06:48 (1767586008) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (2s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 23:06:50 (1767586010) -rw-r--r-- 1 root root 0 Jan 4 23:06 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (2s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 23:06:52 (1767586012) -rw-r--r-- 1 root root 0 Jan 4 23:06 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (2s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 23:06:54 (1767586014) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (2s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 23:06:56 (1767586016) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (2s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 23:06:58 (1767586018) total 1 -rwxrwxrwx 1 root root 0 Jan 4 23:06 f2a -rwxrwxrwx 1 root root 0 Jan 4 23:06 f2b total 1 -rwxrwxrwx 1 root root 0 Jan 4 23:06 f2a -rwxrwxrwx 1 root root 0 Jan 4 23:06 f2b running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.0024547 s, 209 kB/s PASS 2e (2s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 23:07:00 (1767586020) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 23:07:01 (1767586021) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0676556 s, 31.0 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (2s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 23:07:03 (1767586023) PASS 3 (2s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 23:07:05 (1767586025) PASS 4 (4s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 23:07:09 (1767586029) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (2s) == sanityn test 6: remove of open file on other node ============================================================================ 23:07:12 (1767586032) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (3s) == sanityn test 7: remove of open directory on other node ======================================================================= 23:07:14 (1767586034) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (2s) == sanityn test 8: remove of open special file on other node ==================================================================== 23:07:16 (1767586036) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (3s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 23:07:19 (1767586039) PASS 9a (3s) == sanityn test 9b: append to striped sparse file ======== 23:07:22 (1767586042) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0328469 s, 31.9 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.0192347 s, 0.2 kB/s Data read (expecting 'foo'): foo PASS 9b (3s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 23:07:25 (1767586045) 1+0 records in 1+0 records out 1 byte copied, 0.00263347 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00519954 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00495733 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00535775 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00564542 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00480774 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.005065 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00463418 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00519094 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00546519 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00515213 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00516358 s, 0.2 kB/s PASS 10a (3s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 23:07:28 (1767586048) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.00481166 s, 638 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0119615 s, 342 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.000215882 s, 14.2 MB/s PASS 10b (3s) == sanityn test 11: execution of file opened for write should return error ============================================================== 23:07:31 (1767586051) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 300: /mnt/lustre2/d11/f: Text file busy PASS 11 (2s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 23:07:33 (1767586053) warning: '-runtime' deprecated, use '-t runtime' instead - link 1957 (time 1767586064.26 total 10.00 last 195.69) - link 3775 (time 1767586074.26 total 20.00 last 181.76) - link 5117 (time 1767586084.27 total 30.01 last 134.13) - link 6302 (time 1767586094.27 total 40.01 last 118.49) - link 7577 (time 1767586104.27 total 50.02 last 127.40) - link 9562 (time 1767586114.28 total 60.03 last 198.33) - link 10000 (time 1767586117.97 total 63.71 last 118.76) - link 11411 (time 1767586127.98 total 73.72 last 141.05) - link 12592 (time 1767586137.98 total 83.73 last 117.99) - link 13679 (time 1767586147.99 total 93.73 last 108.69) total: 14370 link in 100.00 seconds: 143.69 ops/second using seed 2894600620 running for 100 seconds - stat 10000 (time 1767586064 ; total 9 ; last 9) - stat 20000 (time 1767586073 ; total 18 ; last 9) - stat 27909 (time 1767586084 ; total 29 ; last 11) - stat 30000 (time 1767586087 ; total 32 ; last 3) - stat 38673 (time 1767586098 ; total 43 ; last 11) - stat 40000 (time 1767586099 ; total 44 ; last 1) - stat 47425 (time 1767586110 ; total 55 ; last 11) - stat 50000 (time 1767586117 ; total 62 ; last 7) - stat 53659 (time 1767586128 ; total 73 ; last 11) - stat 57116 (time 1767586139 ; total 84 ; last 11) - stat 60000 (time 1767586148 ; total 93 ; last 9) total: 62387 stats in 100 seconds: 623.869995 stats/second - unlinked 0 (time 1767586105 ; total 0 ; last 0) - unlinked 10000 (time 1767586186 ; total 81 ; last 81) unlink(/mnt/lustre2/lockdir/lockfile14370) error: No such file or directory total: 14369 unlinks in 103 seconds: 139.504852 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (21323) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (21325) - No such process PASS 12 (158s) == sanityn test 13: test directory page revocation ======= 23:10:11 (1767586211) f13 f13 PASS 13 (4s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 23:10:15 (1767586215) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4923: 22513 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 23:10:17 (1767586217) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4923: 23093 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (3s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 23:10:20 (1767586220) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 350: 23671 Terminated $DIR1/$tdir/sleep 60 PASS 14b (3s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 23:10:23 (1767586223) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 362: 24253 Terminated $DIR1/$tdir/sleep 60 PASS 14c (3s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 23:10:26 (1767586226) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 373: 24835 Terminated $DIR1/$tdir/sleep 60 PASS 14d (4s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 23:10:30 (1767586230) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: mgs: Rocky Linux release 8.10 (Green Obsidian) MGS_OS_ID_LIKE=rhel centos fedora rocky MGS_OS_VERSION_ID=8.10 MGS_OS_ID=rocky MGS_OS_VERSION_CODE=134873088 mds1: Rocky Linux release 8.10 (Green Obsidian) MDS1_OS_VERSION_ID=8.10 MDS1_OS_VERSION_CODE=134873088 MDS1_OS_ID_LIKE=rhel centos fedora rocky MDS1_OS_ID=rocky ost1: Rocky Linux release 8.10 (Green Obsidian) OST1_OS_VERSION_CODE=134873088 OST1_OS_ID_LIKE=rhel centos fedora rocky OST1_OS_VERSION_ID=8.10 OST1_OS_ID=rocky client: Rocky Linux release 8.10 (Green Obsidian) CLIENT_OS_ID=rocky CLIENT_OS_VERSION_CODE=134873088 CLIENT_OS_VERSION_ID=8.10 CLIENT_OS_ID_LIKE=rhel centos fedora rocky sleep 5 for ZFS MDS Waiting for MDT destroys to complete fallocate on zfs doesn't consume space STRIPECOUNT=2 ORIGFREE=7524352 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7524352KB free > 800000KB, increase MAXFREE (or reduce fs size) sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (33s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 23:11:03 (1767586263) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.168235 s, 49.9 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.127493 s, 65.8 MB/s Chance of close/open is 1 in 50 Seed set to 6265 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read 000011[0] 1767586265.517224 mapwrite 0x312cbc thru 0x313572 (0x8b7 bytes) truncating to largest ever: 0xb6fe3 000013[0] 1767586265.559286 trunc from 0x313573 to 0x0b6fe2 (0x25c590 bytes) 000015[0] 1767586265.576676 mapread 0x07f335 thru 0x08b3b0 (0xc07c bytes) truncating to largest ever: 0x7fbe63 000016[0] 1767586265.589288 trunc from 0x0b6fe3 to 0x7fbe62 (0x744e80 bytes) 000019[1] 1767586265.609602 mapread 0x4510cd thru 0x454515 (0x3449 bytes) 000020[1] 1767586265.628339 trunc from 0x7fbe63 to 0x2fc5a0 (0x4ff8c2 bytes) 000022[0] 1767586265.645655 read 0x20fa0b thru 0x212c73 (0x3269 bytes) 000023[1] 1767586265.661385 read 0x03e2a9 thru 0x03fbb7 (0x190f bytes) 000027[1] 1767586265.681171 read 0x03860c thru 0x04611a (0xdb0f bytes) 000031[1] 1767586265.731879 write 0x5451ec thru 0x54fe49 (0xac5e bytes) truncating to largest ever: 0x97820b 000033[0] 1767586265.743821 trunc from 0x54fe4a to 0x97820a (0x4283c1 bytes) 000034[0] 1767586265.775124 mapwrite 0x403383 thru 0x412718 (0xf396 bytes) 000036[1] 1767586265.846679 mapread 0x5a55a7 thru 0x5a685e (0x12b8 bytes) 000038[1] 1767586265.873503 read 0x4038b4 thru 0x40a4a4 (0x6bf1 bytes) 000041[0] 1767586265.896350 read 0x900294 thru 0x900921 (0x68e bytes) 000042[0] 1767586265.907392 mapwrite 0x8bbbe7 thru 0x8c7d6e (0xc188 bytes) 000043[1] 1767586265.957659 write 0x70fa1a thru 0x71337c (0x3963 bytes) 000045[1] 1767586265.985061 read 0x8e991f thru 0x8e9c97 (0x379 bytes) 000049[1] 1767586266.016842 read 0x4e1b53 thru 0x4ecec0 (0xb36e bytes) 000054[0] 1767586266.050419 read 0x594e00 thru 0x59cedf (0x80e0 bytes) 000057[0] 1767586266.065327 mapwrite 0x3992b0 thru 0x3a18d3 (0x8624 bytes) 000060[0] 1767586266.125432 mapwrite 0x51a4a1 thru 0x52a2a2 (0xfe02 bytes) 000062[1] 1767586266.184948 trunc from 0x97820b to 0x1e8d7c (0x78f48e bytes) 000063[1] 1767586266.206940 trunc from 0x1e8d7d to 0x677a65 (0x48ece9 bytes) 000064[1] 1767586266.225559 trunc from 0x677a66 to 0x855ca1 (0x1de23c bytes) 000066[1] 1767586266.237123 mapwrite 0x374207 thru 0x37efae (0xada8 bytes) 000069[0] 1767586266.286813 mapread 0x80a8f1 thru 0x819b06 (0xf216 bytes) 000073[0] 1767586266.333074 read 0x287bd7 thru 0x289331 (0x175b bytes) 000074[1] 1767586266.336324 write 0x7d6ea3 thru 0x7e1a21 (0xab7f bytes) 000075[1] 1767586266.350370 trunc from 0x855ca2 to 0x2f6de1 (0x55eec0 bytes) 000076[1] 1767586266.365785 mapread 0x0eb986 thru 0x0f637b (0xa9f6 bytes) 000077[1] 1767586266.381581 write 0x45d78f thru 0x463e2f (0x66a1 bytes) 000078[0] 1767586266.394439 mapread 0x0f5841 thru 0x0f9d42 (0x4502 bytes) 000079[0] 1767586266.417954 write 0x7151ca thru 0x715a2f (0x866 bytes) 000082[0] 1767586266.446437 mapwrite 0x3a3870 thru 0x3b2430 (0xebc1 bytes) 000084[1] 1767586266.535679 write 0x95e720 thru 0x963aeb (0x53cc bytes) 000085[1] 1767586266.550441 trunc from 0x963aec to 0x0dc56b (0x887580 bytes) 000090[0] 1767586266.593095 read 0x007cd9 thru 0x016553 (0xe87b bytes) 000094[1] 1767586266.624707 mapread 0x07fe19 thru 0x088a05 (0x8bed bytes) 000095[1] 1767586266.634804 read 0x0c9db4 thru 0x0d83e8 (0xe635 bytes) 000096[1] 1767586266.655760 mapwrite 0x8c546c thru 0x8c800b (0x2ba0 bytes) 000100[0] 1767586266.814782 write 0x19aae4 thru 0x19f312 (0x482f bytes) 000102[0] 1767586266.892790 write 0x7ef8f0 thru 0x7f9112 (0x9823 bytes) 000104[1] 1767586266.956356 write 0x7b8b01 thru 0x7c415a (0xb65a bytes) 000105[1] 1767586266.987418 mapread 0x817496 thru 0x8205bd (0x9128 bytes) 000107[1] 1767586267.053318 write 0x644d8b thru 0x647fed (0x3263 bytes) 000109[0] 1767586267.076448 mapread 0x33db84 thru 0x33e651 (0xace bytes) 000110[0] 1767586267.096190 mapread 0x76d406 thru 0x774eb0 (0x7aab bytes) 000111[1] 1767586267.135642 read 0x3452e2 thru 0x351938 (0xc657 bytes) 000115[0] 1767586267.142321 trunc from 0x8c800c to 0x1a46ae (0x72395d bytes) 000116[1] 1767586267.163125 mapwrite 0x9f2844 thru 0x9f7bbb (0x5378 bytes) 000117[1] 1767586267.221702 mapread 0x007724 thru 0x01395f (0xc23c bytes) 000118[0] 1767586267.249526 mapwrite 0x6db66f thru 0x6e1a8e (0x6420 bytes) 000120[1] 1767586267.320373 read 0x07dfaf thru 0x086787 (0x87d9 bytes) 000122[1] 1767586267.341143 mapwrite 0x3fcd4d thru 0x404e2f (0x80e3 bytes) 000123[0] 1767586267.419373 mapread 0x78f08c thru 0x798f67 (0x9edc bytes) 000124[1] 1767586267.459263 mapwrite 0x72ede8 thru 0x73d856 (0xea6f bytes) 000125[1] 1767586267.506346 read 0x4bdb8d thru 0x4bfb03 (0x1f77 bytes) 000129[0] 1767586267.537703 trunc from 0x9f7bbc to 0x673f6b (0x383c50 bytes) 000130[0] 1767586267.568527 mapread 0x0de994 thru 0x0e72de (0x894b bytes) 000131[0] 1767586267.605258 trunc from 0x673f6c to 0x7d041e (0x15c4b3 bytes) 000132[0] 1767586267.615719 read 0x569cd4 thru 0x56c721 (0x2a4e bytes) 000133[0] 1767586267.631623 mapread 0x398e5a thru 0x3a263f (0x97e6 bytes) 000136[1] 1767586267.643691 mapread 0x340912 thru 0x34897d (0x806c bytes) 000138[0] 1767586267.667956 mapread 0x3acff0 thru 0x3b0add (0x3aee bytes) 000143[1] 1767586267.684160 mapwrite 0x6d828f thru 0x6e465b (0xc3cd bytes) 000144[1] 1767586267.735706 write 0x31337d thru 0x321dc6 (0xea4a bytes) 000146[0] 1767586267.747693 mapread 0x4627c6 thru 0x46e260 (0xba9b bytes) 000147[0] 1767586267.769116 read 0x39fc1e thru 0x3abaeb (0xbece bytes) 000149[0] 1767586267.784500 trunc from 0x7d041f to 0x3087e3 (0x4c7c3b bytes) 000150[0] 1767586267.814507 mapwrite 0x4d5794 thru 0x4e4422 (0xec8f bytes) 000153[1] 1767586267.889385 mapwrite 0x171222 thru 0x179610 (0x83ef bytes) 000156[0] 1767586267.959248 trunc from 0x4e4423 to 0x247985 (0x29ca9d bytes) 000157[1] 1767586267.974771 write 0x580f40 thru 0x58156a (0x62b bytes) 000158[0] 1767586267.992176 trunc from 0x58156b to 0x3769ae (0x20abbc bytes) 000159[1] 1767586268.011462 mapread 0x1fa410 thru 0x201ca0 (0x7891 bytes) 000160[1] 1767586268.028310 trunc from 0x3769af to 0x38d79e (0x16df0 bytes) 000161[0] 1767586268.057042 read 0x0d26bf thru 0x0d6bea (0x452c bytes) 000166[1] 1767586268.139065 mapread 0x30955b thru 0x30a11b (0xbc1 bytes) 000167[1] 1767586268.154026 read 0x1f32c7 thru 0x202c7f (0xf9b9 bytes) 000169[0] 1767586268.205081 write 0x674196 thru 0x675bea (0x1a55 bytes) 000176[1] 1767586268.239809 read 0x1cf1ec thru 0x1d62cc (0x70e1 bytes) 000177[1] 1767586268.240413 write 0x07ea2b thru 0x087dcd (0x93a3 bytes) 000178[0] 1767586268.277270 trunc from 0x675beb to 0x88ba8c (0x215ea2 bytes) 000183[1] 1767586268.328198 read 0x88590a thru 0x88617e (0x875 bytes) 000184[0] 1767586268.345608 write 0x65a770 thru 0x65cd64 (0x25f5 bytes) 000185[0] 1767586268.353771 read 0x51cb30 thru 0x527682 (0xab53 bytes) 000186[0] 1767586268.358844 read 0x1f2e2b thru 0x1f62c6 (0x349c bytes) 000188[0] 1767586268.380985 mapwrite 0x0d8090 thru 0x0db126 (0x3097 bytes) 000190[0] 1767586268.420344 write 0x8cfdfe thru 0x8d3c8c (0x3e8f bytes) 000192[0] 1767586268.441392 read 0x31a697 thru 0x31ebbf (0x4529 bytes) 000193[1] 1767586268.455111 trunc from 0x8d3c8d to 0x1f357d (0x6e070f bytes) 000194[0] 1767586268.483092 mapwrite 0x959e7a thru 0x960b48 (0x6ccf bytes) 000198[0] 1767586268.562387 mapread 0x70b829 thru 0x715ac6 (0xa29e bytes) 000203[1] 1767586268.603129 mapread 0x8e1d41 thru 0x8e8943 (0x6c03 bytes) 000205[1] 1767586268.635172 mapwrite 0x8f242e thru 0x8f3744 (0x1317 bytes) 000207[0] 1767586268.686327 read 0x1bc01d thru 0x1c7e66 (0xbe4a bytes) 000208[0] 1767586268.696375 mapread 0x8a016d thru 0x8a57f9 (0x568d bytes) 000210[1] 1767586268.716951 trunc from 0x960b49 to 0x0bce4e (0x8a3cfa bytes) 000212[1] 1767586268.738327 write 0x8aaccd thru 0x8abfac (0x12e0 bytes) 000214[0] 1767586268.754014 read 0x15e90e thru 0x162553 (0x3c46 bytes) 000216[1] 1767586268.778257 write 0x8969c1 thru 0x89e3c4 (0x7a04 bytes) 000217[1] 1767586268.788854 mapwrite 0x0bef04 thru 0x0c35ef (0x46ec bytes) 000219[0] 1767586268.829053 read 0x3d6de5 thru 0x3e4d8d (0xdfa9 bytes) 000221[1] 1767586268.862341 mapread 0x0a02a8 thru 0x0ad536 (0xd28f bytes) 000223[1] 1767586268.890291 mapwrite 0x8e4702 thru 0x8edc4c (0x954b bytes) 000224[0] 1767586268.940982 read 0x556852 thru 0x55bdd9 (0x5588 bytes) 000226[1] 1767586268.964047 mapread 0x2356da thru 0x241258 (0xbb7f bytes) 000227[1] 1767586268.979786 mapwrite 0x7b5fbb thru 0x7be420 (0x8466 bytes) 000229[1] 1767586269.022831 write 0x95fa57 thru 0x969134 (0x96de bytes) 000230[0] 1767586269.033635 write 0x4a501a thru 0x4a5f74 (0xf5b bytes) 000231[1] 1767586269.050219 mapwrite 0x4fcfc4 thru 0x50110f (0x414c bytes) 000232[1] 1767586269.105439 trunc from 0x969135 to 0x28f02a (0x6da10a bytes) 000233[0] 1767586269.117553 write 0x7633eb thru 0x76f863 (0xc479 bytes) 000236[1] 1767586269.154254 trunc from 0x76f864 to 0x5a7d5f (0x1c7b04 bytes) 000241[1] 1767586269.194519 read 0x0827f5 thru 0x08bb71 (0x937d bytes) 000242[0] 1767586269.199622 read 0x00581f thru 0x005b05 (0x2e7 bytes) 000243[0] 1767586269.216139 mapwrite 0x3d8c51 thru 0x3e80fb (0xf4ab bytes) 000247[1] 1767586269.303181 mapread 0x430430 thru 0x4358fc (0x54cd bytes) 000248[0] 1767586269.326349 mapwrite 0x5c104a thru 0x5cf27d (0xe234 bytes) 000250[0] 1767586269.392215 write 0x4293cc thru 0x435451 (0xc086 bytes) 000251[1] 1767586269.408398 mapwrite 0x877289 thru 0x877405 (0x17d bytes) 000252[1] 1767586269.455780 trunc from 0x877406 to 0x2c43ad (0x5b3058 bytes) 000253[0] 1767586269.464743 trunc from 0x2c43ae to 0x231054 (0x93359 bytes) 000256[0] 1767586269.484643 mapread 0x1c8720 thru 0x1d63f6 (0xdcd7 bytes) 000257[0] 1767586269.500336 trunc from 0x231055 to 0x83ceff (0x60beab bytes) 000259[0] 1767586269.520278 mapread 0x1f07a9 thru 0x1f7f6c (0x77c4 bytes) 000260[0] 1767586269.525758 trunc from 0x83cf00 to 0x942fc8 (0x1060c9 bytes) 000266[1] 1767586269.538864 write 0x4f6dfe thru 0x506463 (0xf666 bytes) 000267[0] 1767586269.561059 write 0x714258 thru 0x719911 (0x56ba bytes) 000268[1] 1767586269.585498 write 0x9a5300 thru 0x9ac459 (0x715a bytes) 000269[1] 1767586269.601308 mapread 0x111c40 thru 0x119af5 (0x7eb6 bytes) 000274[0] 1767586269.660295 mapread 0x8709ad thru 0x876fa0 (0x65f4 bytes) 000275[1] 1767586269.685091 read 0x95ddef thru 0x964045 (0x6257 bytes) 000278[0] 1767586269.693335 read 0x8389bb thru 0x8450af (0xc6f5 bytes) 000279[0] 1767586269.698686 write 0x76a8ee thru 0x774a5a (0xa16d bytes) 000285[0] 1767586269.724792 mapwrite 0x1dba47 thru 0x1e4adc (0x9096 bytes) 000287[1] 1767586269.775001 read 0x31a01e thru 0x3253ae (0xb391 bytes) 000289[1] 1767586269.803236 read 0x60bd10 thru 0x615bda (0x9ecb bytes) 000290[1] 1767586269.820779 mapwrite 0x6749c4 thru 0x67d0c2 (0x86ff bytes) 000291[0] 1767586269.874158 trunc from 0x9ac45a to 0x0b841b (0x8f403e bytes) 000292[1] 1767586269.894988 write 0x14f05f thru 0x15217e (0x3120 bytes) 000293[0] 1767586269.909421 read 0x0e6e79 thru 0x0e7e6a (0xff2 bytes) 000294[0] 1767586269.937462 mapread 0x0a7c4f thru 0x0b4afa (0xceac bytes) 000295[0] 1767586269.949982 mapwrite 0x0343e4 thru 0x0362e0 (0x1efd bytes) 000299[0] 1767586270.011566 mapread 0x10fbe9 thru 0x110bfa (0x1012 bytes) 000300[0] 1767586270.017829 read 0x0018e2 thru 0x00561b (0x3d3a bytes) 000301[1] 1767586270.046549 read 0x0a1073 thru 0x0ac98b (0xb919 bytes) 000302[1] 1767586270.081881 mapwrite 0x6447d5 thru 0x654010 (0xf83c bytes) 000303[1] 1767586270.185313 write 0x3c964b thru 0x3ca317 (0xccd bytes) 000310[0] 1767586270.209256 trunc from 0x654011 to 0x5eb8a9 (0x68767 bytes) 000312[1] 1767586270.258070 read 0x0c3bab thru 0x0ca861 (0x6cb7 bytes) 000313[1] 1767586270.275684 mapwrite 0x84d0a2 thru 0x859af9 (0xca58 bytes) 000314[1] 1767586270.332520 trunc from 0x859afa to 0x667585 (0x1f2574 bytes) 000315[1] 1767586270.349338 mapwrite 0x8e36ff thru 0x8e8c8a (0x558c bytes) 000318[0] 1767586270.409698 mapwrite 0x1a33be thru 0x1b2014 (0xec57 bytes) 000323[1] 1767586270.529374 trunc from 0x8e8c8b to 0x4479dd (0x4a12ad bytes) 000324[0] 1767586270.559024 trunc from 0x4479de to 0x6c3974 (0x27bf97 bytes) 000325[0] 1767586270.584520 mapread 0x524ae2 thru 0x527709 (0x2c28 bytes) 000326[0] 1767586270.628251 write 0x312dfa thru 0x320961 (0xdb68 bytes) 000328[0] 1767586270.645327 trunc from 0x6c3975 to 0x28813c (0x43b838 bytes) 000330[0] 1767586270.670347 write 0x6e8a8a thru 0x6f2cd4 (0xa24b bytes) 000332[0] 1767586270.681773 trunc from 0x6f2cd5 to 0x08b693 (0x667641 bytes) 000334[1] 1767586270.702179 mapwrite 0x9fb6bb thru 0x9fffff (0x4945 bytes) 000336[1] 1767586270.768987 trunc from 0xa00000 to 0x208dfb (0x7f7204 bytes) truncating to largest ever: 0x97b87b 000337[0] 1767586270.779694 trunc from 0x208dfc to 0x97b87a (0x772a7f bytes) 000338[0] 1767586270.799320 write 0x5503f2 thru 0x55cc0e (0xc81d bytes) 000339[0] 1767586270.816066 read 0x0df1e4 thru 0x0e777c (0x8599 bytes) 000340[0] 1767586270.821222 write 0x679af4 thru 0x67e02a (0x4537 bytes) 000341[0] 1767586270.827559 trunc from 0x97b87b to 0x4b5ba5 (0x4c5cd5 bytes) 000343[1] 1767586270.850322 trunc from 0x4b5ba6 to 0x888695 (0x3d2af0 bytes) 000344[1] 1767586270.878349 write 0x85a7cf thru 0x865e01 (0xb633 bytes) 000348[0] 1767586270.898342 trunc from 0x888696 to 0x5202b8 (0x3683dd bytes) 000350[1] 1767586270.934469 mapread 0x06fb77 thru 0x07038b (0x815 bytes) 000351[1] 1767586270.952344 mapwrite 0x5aa2d5 thru 0x5b3b8b (0x98b7 bytes) 000352[0] 1767586271.018355 read 0x324393 thru 0x332f44 (0xebb2 bytes) 000353[0] 1767586271.033238 read 0x4dbbc0 thru 0x4e748f (0xb8d0 bytes) 000355[0] 1767586271.048838 trunc from 0x5b3b8c to 0x88db61 (0x2d9fd6 bytes) 000356[1] 1767586271.069677 trunc from 0x88db62 to 0x474337 (0x41982a bytes) 000357[0] 1767586271.090243 mapread 0x0c5132 thru 0x0d1c1c (0xcaeb bytes) 000360[0] 1767586271.137665 write 0x30159e thru 0x3054fa (0x3f5d bytes) 000363[1] 1767586271.174747 read 0x2ca1f4 thru 0x2cffc0 (0x5dcd bytes) 000364[1] 1767586271.197357 read 0x2dbb1d thru 0x2dcd09 (0x11ed bytes) 000365[1] 1767586271.209718 mapwrite 0x8db3df thru 0x8e84ca (0xd0ec bytes) 000368[1] 1767586271.286356 mapread 0x348a2c thru 0x349ec9 (0x149e bytes) 000369[1] 1767586271.302113 mapread 0x372257 thru 0x379d70 (0x7b1a bytes) 000370[1] 1767586271.316285 trunc from 0x8e84cb to 0x7c0522 (0x127fa8 bytes) 000372[0] 1767586271.327876 write 0x47c8be thru 0x47d0e6 (0x829 bytes) 000373[1] 1767586271.348566 mapwrite 0x21c87e thru 0x228afe (0xc281 bytes) 000374[0] 1767586271.399337 write 0x40b19a thru 0x410e2f (0x5c96 bytes) 000377[0] 1767586271.431661 write 0x04abd2 thru 0x057d13 (0xd142 bytes) 000379[0] 1767586271.462465 trunc from 0x7c0523 to 0x8f309e (0x132b7c bytes) 000381[1] 1767586271.485689 mapread 0x5069a5 thru 0x50b541 (0x4b9d bytes) 000384[0] 1767586271.529204 read 0x80f9f4 thru 0x81408b (0x4698 bytes) 000387[1] 1767586271.559241 read 0x548713 thru 0x54dbaa (0x5498 bytes) 000388[1] 1767586271.572268 mapread 0x7486c2 thru 0x7574db (0xee1a bytes) 000390[1] 1767586271.592590 write 0x1cca31 thru 0x1d0066 (0x3636 bytes) 000392[0] 1767586271.616897 mapread 0x86fe53 thru 0x87d936 (0xdae4 bytes) 000393[0] 1767586271.646600 mapwrite 0x57d1eb thru 0x589fef (0xce05 bytes) 000399[1] 1767586271.721435 mapwrite 0x6402bb thru 0x6467e4 (0x652a bytes) 000401[1] 1767586271.782572 mapwrite 0x2717a1 thru 0x280fe8 (0xf848 bytes) 000402[1] 1767586271.838710 write 0x627715 thru 0x62fc84 (0x8570 bytes) 000403[0] 1767586271.853319 trunc from 0x8f309f to 0x73cfce (0x1b60d0 bytes) 000405[1] 1767586271.888435 mapwrite 0x3a870e thru 0x3aa43b (0x1d2e bytes) 000406[0] 1767586271.936601 mapread 0x043b94 thru 0x051e48 (0xe2b5 bytes) 000409[1] 1767586271.970090 read 0x3c879a thru 0x3d01ef (0x7a56 bytes) 000410[1] 1767586271.983183 trunc from 0x73cfcf to 0x77a10a (0x3d13c bytes) 000411[1] 1767586272.010323 trunc from 0x77a10b to 0x3b0e6e (0x3c929c bytes) 000412[0] 1767586272.029973 read 0x001683 thru 0x002f96 (0x1914 bytes) 000413[0] 1767586272.078238 mapwrite 0x8ca837 thru 0x8d365f (0x8e29 bytes) 000421[0] 1767586272.239239 trunc from 0x8d3660 to 0x8c74bc (0xc1a3 bytes) 000423[1] 1767586272.250036 read 0x40f44f thru 0x41c503 (0xd0b5 bytes) 000425[0] 1767586272.285896 mapwrite 0x08b372 thru 0x097af5 (0xc784 bytes) 000426[0] 1767586272.350867 read 0x376542 thru 0x37e598 (0x8057 bytes) 000427[1] 1767586272.356375 read 0x0e0559 thru 0x0e0e22 (0x8ca bytes) 000430[0] 1767586272.369885 write 0x993732 thru 0x99f5a8 (0xbe77 bytes) 000431[0] 1767586272.377287 mapread 0x64072c thru 0x644d3f (0x4614 bytes) 000433[1] 1767586272.406278 mapread 0x00e9ae thru 0x013292 (0x48e5 bytes) 000434[1] 1767586272.431744 write 0x3c135e thru 0x3cea07 (0xd6aa bytes) 000436[0] 1767586272.461284 read 0x060832 thru 0x069a6a (0x9239 bytes) 000441[0] 1767586272.479569 read 0x5c5c18 thru 0x5d5a25 (0xfe0e bytes) 000442[0] 1767586272.484985 write 0x84c35a thru 0x8549eb (0x8692 bytes) 000443[0] 1767586272.495862 mapwrite 0x0de088 thru 0x0e6a8f (0x8a08 bytes) 000444[1] 1767586272.544187 mapread 0x2e559f thru 0x2eb885 (0x62e7 bytes) 000445[1] 1767586272.558418 write 0x68ed9e thru 0x690724 (0x1987 bytes) 000446[0] 1767586272.578210 mapread 0x881a42 thru 0x8886d4 (0x6c93 bytes) 000447[0] 1767586272.602161 mapwrite 0x482634 thru 0x48eb04 (0xc4d1 bytes) 000451[0] 1767586272.682693 write 0x239de3 thru 0x241142 (0x7360 bytes) 000452[1] 1767586272.694571 mapread 0x69fe5a thru 0x6add1c (0xdec3 bytes) 000457[1] 1767586272.731561 mapread 0x71b820 thru 0x72ab02 (0xf2e3 bytes) 000458[0] 1767586272.755142 write 0x9fc049 thru 0x9fee98 (0x2e50 bytes) 000461[1] 1767586272.765889 trunc from 0x9fee99 to 0x0c4343 (0x93ab55 bytes) 000462[1] 1767586272.804364 mapwrite 0x560312 thru 0x562c66 (0x2955 bytes) 000464[1] 1767586272.852538 mapwrite 0x9e5313 thru 0x9ee52e (0x921c bytes) 000466[1] 1767586272.912440 mapread 0x80858c thru 0x81835b (0xfdd0 bytes) 000468[0] 1767586272.938462 trunc from 0x9ee52f to 0x24efe3 (0x79f54b bytes) 000470[0] 1767586272.978338 write 0x1bab34 thru 0x1bedac (0x4279 bytes) 000473[0] 1767586272.996276 trunc from 0x24efe4 to 0x77655e (0x52757b bytes) 000476[1] 1767586273.036595 mapread 0x753f57 thru 0x756551 (0x25fb bytes) 000481[0] 1767586273.083703 read 0x5142f7 thru 0x5172e1 (0x2feb bytes) 000482[0] 1767586273.087352 trunc from 0x77655f to 0x6483fe (0x12e160 bytes) 000484[0] 1767586273.114286 trunc from 0x6483ff to 0x6a8bc5 (0x607c7 bytes) 000488[1] 1767586273.137000 mapread 0x341c1d thru 0x34c965 (0xad49 bytes) 000490[0] 1767586273.169833 trunc from 0x6a8bc6 to 0x6103c8 (0x987fd bytes) 000491[0] 1767586273.186674 mapwrite 0x503053 thru 0x512b05 (0xfab3 bytes) 000492[0] 1767586273.240800 mapread 0x5ed2b5 thru 0x5f22a0 (0x4fec bytes) 000494[0] 1767586273.259326 read 0x22abd5 thru 0x23a9bf (0xfdeb bytes) 000495[0] 1767586273.264748 read 0x14dbeb thru 0x14f22d (0x1643 bytes) 000496[0] 1767586273.268064 mapread 0x2cc3aa thru 0x2d43ec (0x8043 bytes) 000499[1] 1767586273.306942 write 0x6dab9c thru 0x6e0601 (0x5a66 bytes) 000500[1] 1767586273.323265 mapread 0x3bd241 thru 0x3c8c0c (0xb9cc bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 6273 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[0] 1767586273.691516 mapwrite 0x145000 thru 0x1494c4 (0x44c5 bytes) 000003[0] 1767586273.750247 read 0x032000 thru 0x03908a (0x708b bytes) 000004[1] 1767586273.763075 read_OD 0x060000 thru 0x064fff (0x5000 bytes) truncating to largest ever: 0xf9b6f 000005[0] 1767586273.777352 trunc from 0x1494c5 to 0x0f9b6e (0x4f956 bytes) 000006[0] 1767586273.791738 write 0x6da000 thru 0x6ddd90 (0x3d91 bytes) 000009[0] 1767586273.813132 mapread 0x116000 thru 0x118b80 (0x2b81 bytes) 000011[1] 1767586273.835250 write_OD 0x080000 thru 0x08dfff (0xe000 bytes) 000014[0] 1767586273.868157 read 0x61d000 thru 0x62c91d (0xf91e bytes) 000015[1] 1767586273.873584 mapwrite 0x7e3000 thru 0x7e567f (0x2680 bytes) truncating to largest ever: 0x34cd9f 000016[0] 1767586273.919697 trunc from 0x7e5680 to 0x34cd9e (0x4988e1 bytes) 000018[1] 1767586273.954834 mapread 0x2be000 thru 0x2bfbdb (0x1bdc bytes) 000019[0] 1767586273.970298 mapread 0x0d5000 thru 0x0dd250 (0x8251 bytes) 000020[1] 1767586273.984889 mapwrite 0x6b7000 thru 0x6bc3cd (0x53ce bytes) 000023[1] 1767586274.034826 mapwrite 0x25a000 thru 0x26660a (0xc60b bytes) 000025[1] 1767586274.097189 read_OD 0x259000 thru 0x263fff (0xb000 bytes) truncating to largest ever: 0x815ea9 000026[1] 1767586274.107311 trunc from 0x6bc3ce to 0x815ea8 (0x159adb bytes) 000029[1] 1767586274.147212 mapread 0x38e000 thru 0x39d66c (0xf66d bytes) 000031[1] 1767586274.174344 write_OD 0x289000 thru 0x296fff (0xe000 bytes) 000034[0] 1767586274.204529 trunc from 0x815ea9 to 0x72c106 (0xe9da2 bytes) 000035[1] 1767586274.219560 trunc from 0x72c107 to 0x6239de (0x108728 bytes) 000036[0] 1767586274.233561 read 0x029000 thru 0x033136 (0xa137 bytes) truncating to largest ever: 0x8e07ac 000038[1] 1767586274.248350 trunc from 0x6239df to 0x8e07ab (0x2bcdcd bytes) 000042[1] 1767586274.292169 mapwrite 0x21e000 thru 0x226082 (0x8083 bytes) 000047[0] 1767586274.360561 write 0x24a000 thru 0x2573f3 (0xd3f4 bytes) 000049[1] 1767586274.387155 trunc from 0x8e07ac to 0x21ae05 (0x6c59a6 bytes) 000051[0] 1767586274.416668 read 0x068000 thru 0x074906 (0xc907 bytes) 000053[1] 1767586274.458979 mapwrite 0x55c000 thru 0x566b11 (0xab12 bytes) 000058[0] 1767586274.535772 mapwrite 0x617000 thru 0x61d3a2 (0x63a3 bytes) 000062[1] 1767586274.646879 mapwrite 0x028000 thru 0x02dde7 (0x5de8 bytes) 000070[1] 1767586274.730195 write_OD 0x1d1000 thru 0x1d5fff (0x5000 bytes) 000071[1] 1767586274.755042 mapread 0x513000 thru 0x519b3d (0x6b3e bytes) 000074[0] 1767586274.768441 mapread 0x52e000 thru 0x53b688 (0xd689 bytes) 000075[0] 1767586274.808621 write 0x3a4000 thru 0x3a5921 (0x1922 bytes) 000078[1] 1767586274.818271 read_OD 0x5ba000 thru 0x5bdfff (0x4000 bytes) 000082[1] 1767586274.836989 read_OD 0x03e000 thru 0x045fff (0x8000 bytes) 000083[1] 1767586274.841410 read_OD 0x562000 thru 0x563fff (0x2000 bytes) 000086[0] 1767586274.845384 mapread 0x2f0000 thru 0x2f8577 (0x8578 bytes) 000089[0] 1767586274.871664 trunc from 0x61d3a3 to 0x63bf98 (0x1ebf6 bytes) 000090[1] 1767586274.887261 read_OD 0x38f000 thru 0x390fff (0x2000 bytes) 000092[1] 1767586274.906473 write_OD 0x071000 thru 0x077fff (0x7000 bytes) 000096[1] 1767586274.939889 mapread 0x233000 thru 0x23f9e2 (0xc9e3 bytes) 000098[0] 1767586274.952981 trunc from 0x63bf99 to 0x8c5627 (0x28968f bytes) 000100[1] 1767586274.971312 read_OD 0x3ca000 thru 0x3ccfff (0x3000 bytes) 000101[1] 1767586274.980640 mapread 0x183000 thru 0x18cb67 (0x9b68 bytes) 000102[0] 1767586274.990689 read 0x5b5000 thru 0x5b8f6c (0x3f6d bytes) 000103[1] 1767586274.993186 write_OD 0x684000 thru 0x687fff (0x4000 bytes) 000106[1] 1767586275.011780 mapread 0x36c000 thru 0x373e5a (0x7e5b bytes) 000107[0] 1767586275.019527 read 0x2c1000 thru 0x2c6943 (0x5944 bytes) 000109[0] 1767586275.023376 mapread 0x332000 thru 0x337ba7 (0x5ba8 bytes) 000110[1] 1767586275.029655 write_OD 0x664000 thru 0x66bfff (0x8000 bytes) 000111[0] 1767586275.048624 read 0x209000 thru 0x216c8a (0xdc8b bytes) 000114[0] 1767586275.053496 mapread 0x56b000 thru 0x57a790 (0xf791 bytes) 000116[1] 1767586275.073771 mapread 0x1bb000 thru 0x1c6e90 (0xbe91 bytes) 000118[0] 1767586275.090147 write 0x3b9000 thru 0x3b952c (0x52d bytes) 000120[1] 1767586275.100881 read_OD 0x0cd000 thru 0x0d6fff (0xa000 bytes) 000121[1] 1767586275.117096 mapread 0x4a7000 thru 0x4b1869 (0xa86a bytes) 000124[1] 1767586275.131236 read_OD 0x001000 thru 0x00afff (0xa000 bytes) 000126[1] 1767586275.136668 mapread 0x518000 thru 0x525c95 (0xdc96 bytes) 000128[0] 1767586275.152643 trunc from 0x8c5628 to 0x735d2e (0x18f8f9 bytes) 000129[1] 1767586275.167251 mapread 0x5c8000 thru 0x5cb30f (0x3310 bytes) 000132[1] 1767586275.184149 read_OD 0x643000 thru 0x646fff (0x4000 bytes) 000133[0] 1767586275.190087 read 0x3df000 thru 0x3e954c (0xa54d bytes) 000135[0] 1767586275.195670 mapwrite 0x3ee000 thru 0x3f1383 (0x3384 bytes) 000137[0] 1767586275.240046 trunc from 0x735d2f to 0x23d5b3 (0x4f877b bytes) 000140[1] 1767586275.269812 trunc from 0x23d5b4 to 0x873d4b (0x636798 bytes) 000142[0] 1767586275.296249 mapwrite 0x5ac000 thru 0x5afbb5 (0x3bb6 bytes) 000144[0] 1767586275.345418 mapread 0x1aa000 thru 0x1ae44f (0x4450 bytes) 000146[0] 1767586275.352222 read 0x5b2000 thru 0x5c0385 (0xe386 bytes) 000150[0] 1767586275.358620 mapread 0x5d4000 thru 0x5d4a1b (0xa1c bytes) 000151[1] 1767586275.360668 read_OD 0x27d000 thru 0x284fff (0x8000 bytes) 000154[0] 1767586275.370786 mapwrite 0x666000 thru 0x66b80a (0x580b bytes) 000155[0] 1767586275.397281 read 0x726000 thru 0x72cac8 (0x6ac9 bytes) 000156[0] 1767586275.400154 read 0x30e000 thru 0x31d96a (0xf96b bytes) 000157[1] 1767586275.403260 mapwrite 0x3da000 thru 0x3e2eba (0x8ebb bytes) 000158[1] 1767586275.439447 read_OD 0x098000 thru 0x0a0fff (0x9000 bytes) 000159[0] 1767586275.445472 trunc from 0x873d4c to 0x7fff77 (0x73dd4 bytes) 000160[0] 1767586275.456281 write 0x5e0000 thru 0x5e70a6 (0x70a7 bytes) 000162[0] 1767586275.466907 read 0x0e7000 thru 0x0f62d2 (0xf2d3 bytes) 000164[1] 1767586275.475416 trunc from 0x7fff78 to 0x5b7e5e (0x248119 bytes) 000165[0] 1767586275.488277 write 0x50b000 thru 0x518eb1 (0xdeb2 bytes) 000167[0] 1767586275.507717 read 0x148000 thru 0x1505bc (0x85bd bytes) 000169[0] 1767586275.521982 write 0x5bd000 thru 0x5c3c21 (0x6c22 bytes) 000170[1] 1767586275.522559 mapread 0x564000 thru 0x565212 (0x1213 bytes) 000173[1] 1767586275.534058 read_OD 0x473000 thru 0x47dfff (0xb000 bytes) 000174[0] 1767586275.537398 mapread 0x456000 thru 0x46525f (0xf260 bytes) 000175[1] 1767586275.545156 read_OD 0x103000 thru 0x10dfff (0xb000 bytes) 000176[0] 1767586275.547360 mapread 0x42a000 thru 0x42d323 (0x3324 bytes) 000177[1] 1767586275.550466 mapwrite 0x2fc000 thru 0x307059 (0xb05a bytes) 000179[1] 1767586275.580815 trunc from 0x5c3c22 to 0x09a859 (0x5293c8 bytes) 000182[0] 1767586275.590545 trunc from 0x09a85a to 0x5aa0c4 (0x50f86b bytes) 000183[1] 1767586275.597634 mapwrite 0x821000 thru 0x82a7c3 (0x97c4 bytes) 000185[0] 1767586275.633378 write 0x8f6000 thru 0x90538b (0xf38c bytes) 000187[0] 1767586275.644400 trunc from 0x90538c to 0x78168c (0x183cff bytes) 000191[1] 1767586275.675324 mapread 0x3ad000 thru 0x3b76ba (0xa6bb bytes) 000192[1] 1767586275.696011 mapwrite 0x603000 thru 0x610f49 (0xdf4a bytes) 000196[1] 1767586275.766613 mapread 0x3f7000 thru 0x405fa3 (0xefa4 bytes) 000199[1] 1767586275.804791 read_OD 0x353000 thru 0x356fff (0x4000 bytes) 000200[0] 1767586275.812830 mapread 0x422000 thru 0x42fb79 (0xdb7a bytes) 000205[0] 1767586275.825435 mapread 0x033000 thru 0x042461 (0xf462 bytes) 000206[0] 1767586275.837616 mapwrite 0x6a3000 thru 0x6a8f1e (0x5f1f bytes) 000207[0] 1767586275.873841 write 0x9f9000 thru 0x9ffbf9 (0x6bfa bytes) 000208[1] 1767586275.875640 read_OD 0x516000 thru 0x51efff (0x9000 bytes) 000209[1] 1767586275.892759 mapread 0x4b3000 thru 0x4c2bd9 (0xfbda bytes) 000210[0] 1767586275.904589 write 0x2e4000 thru 0x2f221f (0xe220 bytes) 000213[1] 1767586275.928650 trunc from 0x9ffbfa to 0x73c21e (0x2c39db bytes) 000215[0] 1767586275.946645 trunc from 0x73c21f to 0x1609ec (0x5db832 bytes) 000216[0] 1767586275.959467 write 0x902000 thru 0x909e9d (0x7e9e bytes) 000220[0] 1767586275.974061 trunc from 0x909e9e to 0x821b61 (0xe833c bytes) 000221[1] 1767586275.984214 mapread 0x274000 thru 0x2750be (0x10bf bytes) 000224[0] 1767586276.003138 mapread 0x645000 thru 0x64767c (0x267d bytes) 000225[1] 1767586276.013567 mapread 0x697000 thru 0x697af4 (0xaf5 bytes) 000227[1] 1767586276.017093 write_OD 0x114000 thru 0x116fff (0x3000 bytes) 000230[1] 1767586276.045167 mapwrite 0x4a0000 thru 0x4a9603 (0x9604 bytes) 000231[0] 1767586276.093026 trunc from 0x821b62 to 0x62e484 (0x1f36dd bytes) 000235[1] 1767586276.126019 write_OD 0x77f000 thru 0x78bfff (0xd000 bytes) 000236[0] 1767586276.151468 mapwrite 0x99e000 thru 0x9a3dd9 (0x5dda bytes) 000237[0] 1767586276.206143 write 0x421000 thru 0x4271a1 (0x61a2 bytes) 000239[0] 1767586276.220586 write 0x78b000 thru 0x793811 (0x8812 bytes) 000240[1] 1767586276.232109 mapread 0x348000 thru 0x350e03 (0x8e04 bytes) 000241[0] 1767586276.258269 trunc from 0x9a3dda to 0x07ff41 (0x923e98 bytes) 000243[1] 1767586276.284003 mapread 0x02b000 thru 0x036860 (0xb861 bytes) 000244[1] 1767586276.308136 trunc from 0x07ff42 to 0x695e6a (0x615f29 bytes) truncating to largest ever: 0x8e4e96 000246[1] 1767586276.323668 trunc from 0x695e6b to 0x8e4e95 (0x24f02b bytes) 000249[0] 1767586276.355041 mapwrite 0x36b000 thru 0x36f6b0 (0x46b1 bytes) 000250[0] 1767586276.408093 read 0x189000 thru 0x18f4c1 (0x64c2 bytes) 000251[1] 1767586276.421012 trunc from 0x8e4e96 to 0x3be46d (0x526a28 bytes) 000252[0] 1767586276.444085 mapread 0x096000 thru 0x0a4a37 (0xea38 bytes) 000253[0] 1767586276.474017 mapread 0x245000 thru 0x24f0b9 (0xa0ba bytes) 000257[1] 1767586276.485531 trunc from 0x3be46e to 0x1ff47f (0x1befee bytes) 000260[1] 1767586276.511182 mapwrite 0x98e000 thru 0x99ad07 (0xcd08 bytes) 000266[0] 1767586276.584060 write 0x168000 thru 0x16ae5c (0x2e5d bytes) 000267[1] 1767586276.594168 read_OD 0x579000 thru 0x579fff (0x1000 bytes) 000268[0] 1767586276.609374 mapread 0x83a000 thru 0x844442 (0xa443 bytes) 000269[0] 1767586276.631521 write 0x6b8000 thru 0x6c1996 (0x9997 bytes) 000271[0] 1767586276.642171 write 0x589000 thru 0x58ac10 (0x1c11 bytes) 000273[1] 1767586276.662611 trunc from 0x99ad08 to 0x06402d (0x936cda bytes) 000274[0] 1767586276.690090 mapread 0x03b000 thru 0x03fd29 (0x4d2a bytes) 000276[1] 1767586276.714297 write_OD 0x00f000 thru 0x00ffff (0x1000 bytes) 000280[0] 1767586276.742367 write 0x112000 thru 0x11cbea (0xabeb bytes) 000281[0] 1767586276.757079 mapread 0x0b2000 thru 0x0bbbb2 (0x9bb3 bytes) 000282[1] 1767586276.775088 mapread 0x00a000 thru 0x00aa4b (0xa4c bytes) 000283[1] 1767586276.791190 write_OD 0x14c000 thru 0x158fff (0xd000 bytes) 000287[0] 1767586276.819557 write 0x1e4000 thru 0x1e75b7 (0x35b8 bytes) 000291[1] 1767586276.835064 mapwrite 0x593000 thru 0x5946e8 (0x16e9 bytes) 000294[1] 1767586276.898010 mapwrite 0x42c000 thru 0x439b23 (0xdb24 bytes) truncating to largest ever: 0x9bc612 000295[0] 1767586276.939005 trunc from 0x5946e9 to 0x9bc611 (0x427f29 bytes) 000297[1] 1767586276.971223 mapread 0x6bf000 thru 0x6c8aec (0x9aed bytes) 000298[0] 1767586276.995208 write 0x564000 thru 0x572f18 (0xef19 bytes) 000299[1] 1767586277.003988 mapwrite 0x948000 thru 0x951ed7 (0x9ed8 bytes) 000301[1] 1767586277.063283 mapread 0x107000 thru 0x10f3fb (0x83fc bytes) 000302[1] 1767586277.082037 mapread 0x8d3000 thru 0x8dee98 (0xbe99 bytes) 000303[0] 1767586277.097244 mapread 0x14c000 thru 0x158e2a (0xce2b bytes) 000305[1] 1767586277.122127 mapwrite 0x796000 thru 0x7a4dbe (0xedbf bytes) 000307[1] 1767586277.188011 mapread 0x67f000 thru 0x68612c (0x712d bytes) 000308[0] 1767586277.206590 mapwrite 0x490000 thru 0x4928e6 (0x28e7 bytes) 000309[1] 1767586277.246259 read_OD 0x1ec000 thru 0x1f1fff (0x6000 bytes) 000311[1] 1767586277.260638 write_OD 0x4fc000 thru 0x506fff (0xb000 bytes) 000313[0] 1767586277.289053 mapwrite 0x670000 thru 0x67ac4e (0xac4f bytes) 000318[0] 1767586277.355042 mapwrite 0x227000 thru 0x230f99 (0x9f9a bytes) 000319[1] 1767586277.389993 mapwrite 0x9df000 thru 0x9e1a55 (0x2a56 bytes) 000320[0] 1767586277.434503 mapwrite 0x811000 thru 0x812aca (0x1acb bytes) 000322[1] 1767586277.473551 write_OD 0x75c000 thru 0x769fff (0xe000 bytes) 000324[0] 1767586277.503462 write 0x46d000 thru 0x46e013 (0x1014 bytes) 000327[0] 1767586277.528026 mapwrite 0x81a000 thru 0x8205ff (0x6600 bytes) 000330[1] 1767586277.562972 mapread 0x2e2000 thru 0x2ebc7b (0x9c7c bytes) 000331[0] 1767586277.583464 read 0x4c7000 thru 0x4cf810 (0x8811 bytes) 000332[0] 1767586277.590896 write 0x8db000 thru 0x8e86c4 (0xd6c5 bytes) 000333[0] 1767586277.609060 read 0x646000 thru 0x64e7f9 (0x87fa bytes) 000334[1] 1767586277.622018 write_OD 0x159000 thru 0x15bfff (0x3000 bytes) 000335[1] 1767586277.651578 mapread 0x475000 thru 0x47a74c (0x574d bytes) 000337[0] 1767586277.664647 trunc from 0x9e1a56 to 0x198a9e (0x848fb7 bytes) 000338[1] 1767586277.680174 read_OD 0x11d000 thru 0x12afff (0xe000 bytes) 000343[0] 1767586277.700476 mapwrite 0x069000 thru 0x06c6c4 (0x36c5 bytes) 000346[1] 1767586277.749540 mapwrite 0x870000 thru 0x87e35d (0xe35e bytes) 000351[0] 1767586277.827753 write 0x01a000 thru 0x029c81 (0xfc82 bytes) 000353[0] 1767586277.846064 trunc from 0x87e35e to 0x79f9ba (0xde9a3 bytes) 000354[1] 1767586277.856053 read_OD 0x3d0000 thru 0x3d7fff (0x8000 bytes) 000355[1] 1767586277.876511 mapread 0x3f1000 thru 0x3f6861 (0x5862 bytes) 000356[1] 1767586277.889478 read_OD 0x63d000 thru 0x646fff (0xa000 bytes) 000358[0] 1767586277.894436 read 0x28a000 thru 0x29586f (0xb870 bytes) 000360[1] 1767586277.899640 trunc from 0x79f9bb to 0x80debb (0x6e501 bytes) 000362[1] 1767586277.914531 mapwrite 0x85c000 thru 0x865bf8 (0x9bf9 bytes) 000363[0] 1767586277.958827 read 0x373000 thru 0x37bb36 (0x8b37 bytes) 000366[1] 1767586277.974522 mapread 0x348000 thru 0x35537e (0xd37f bytes) 000367[0] 1767586277.990125 write 0x694000 thru 0x6a1064 (0xd065 bytes) 000369[0] 1767586278.008365 mapread 0x0fd000 thru 0x109b9a (0xcb9b bytes) 000370[1] 1767586278.022405 read_OD 0x3ec000 thru 0x3f9fff (0xe000 bytes) 000371[1] 1767586278.033514 mapread 0x761000 thru 0x77017e (0xf17f bytes) 000373[1] 1767586278.043809 read_OD 0x158000 thru 0x164fff (0xd000 bytes) 000374[0] 1767586278.046874 mapread 0x44f000 thru 0x4580c2 (0x90c3 bytes) 000375[0] 1767586278.053116 write 0x64f000 thru 0x655f02 (0x6f03 bytes) 000377[0] 1767586278.059856 trunc from 0x865bf9 to 0x136451 (0x72f7a7 bytes) 000378[1] 1767586278.067838 mapwrite 0x0b1000 thru 0x0b79c2 (0x69c3 bytes) 000380[1] 1767586278.123162 write_OD 0x34d000 thru 0x357fff (0xb000 bytes) 000382[0] 1767586278.151491 mapread 0x324000 thru 0x3300c7 (0xc0c8 bytes) 000386[0] 1767586278.170400 read 0x2ed000 thru 0x2ed2c9 (0x2ca bytes) 000390[0] 1767586278.209393 read 0x212000 thru 0x21e8d5 (0xc8d6 bytes) 000393[0] 1767586278.215186 write 0x4c0000 thru 0x4cb7bd (0xb7be bytes) 000395[1] 1767586278.237333 mapwrite 0x924000 thru 0x9330f3 (0xf0f4 bytes) 000396[1] 1767586278.309718 trunc from 0x9330f4 to 0x0ac9ae (0x886745 bytes) 000398[1] 1767586278.329025 mapread 0x028000 thru 0x032ab2 (0xaab3 bytes) 000399[0] 1767586278.348066 mapwrite 0x242000 thru 0x243c85 (0x1c86 bytes) 000402[1] 1767586278.425042 trunc from 0x243c86 to 0x832411 (0x5ee78c bytes) 000406[1] 1767586278.464323 read_OD 0x746000 thru 0x74dfff (0x8000 bytes) 000407[1] 1767586278.474181 trunc from 0x832412 to 0x1a5b79 (0x68c898 bytes) 000412[0] 1767586278.516454 mapwrite 0x501000 thru 0x5036e5 (0x26e6 bytes) 000413[1] 1767586278.593017 mapread 0x433000 thru 0x441b30 (0xeb31 bytes) 000414[0] 1767586278.624383 trunc from 0x5036e6 to 0x785fa7 (0x2828c2 bytes) 000415[0] 1767586278.637213 trunc from 0x785fa8 to 0x54003b (0x245f6c bytes) 000416[1] 1767586278.653165 mapread 0x072000 thru 0x077aba (0x5abb bytes) 000419[1] 1767586278.678488 trunc from 0x54003c to 0x681aac (0x141a71 bytes) 000423[0] 1767586278.704985 write 0x677000 thru 0x6788d8 (0x18d9 bytes) 000424[1] 1767586278.739246 trunc from 0x681aad to 0x2361d3 (0x44b8d9 bytes) 000425[1] 1767586278.758494 write_OD 0x082000 thru 0x088fff (0x7000 bytes) 000426[0] 1767586278.784881 mapwrite 0x626000 thru 0x626d18 (0xd19 bytes) 000430[0] 1767586278.850368 read 0x0c2000 thru 0x0ce706 (0xc707 bytes) 000432[1] 1767586278.855574 trunc from 0x626d19 to 0x14072a (0x4e65ee bytes) 000436[1] 1767586278.889527 read_OD 0x11d000 thru 0x125fff (0x9000 bytes) 000438[1] 1767586278.901266 trunc from 0x14072b to 0x33d8b1 (0x1fd187 bytes) 000439[0] 1767586278.924372 read 0x04e000 thru 0x05a425 (0xc426 bytes) 000441[1] 1767586278.971209 read_OD 0x1ed000 thru 0x1f5fff (0x9000 bytes) 000442[1] 1767586278.975772 write_OD 0x9a8000 thru 0x9acfff (0x5000 bytes) 000443[1] 1767586279.009319 mapread 0x8a8000 thru 0x8ac9c8 (0x49c9 bytes) 000444[0] 1767586279.129569 mapread 0x303000 thru 0x30dcd0 (0xacd1 bytes) 000446[0] 1767586279.168961 mapread 0x147000 thru 0x156506 (0xf507 bytes) 000447[1] 1767586279.212215 mapwrite 0x032000 thru 0x034d3b (0x2d3c bytes) 000448[0] 1767586279.276890 write 0x2df000 thru 0x2eb254 (0xc255 bytes) 000449[1] 1767586279.294896 trunc from 0x9ad000 to 0x29749b (0x715b64 bytes) 000453[0] 1767586279.347562 trunc from 0x29749c to 0x1031b7 (0x1942e4 bytes) 000458[0] 1767586279.378196 trunc from 0x1031b8 to 0x784740 (0x681589 bytes) 000459[0] 1767586279.390407 mapread 0x33e000 thru 0x342855 (0x4856 bytes) 000460[0] 1767586279.415507 write 0x5fd000 thru 0x6092a4 (0xc2a5 bytes) 000463[1] 1767586279.462353 mapwrite 0x4d0000 thru 0x4d4885 (0x4886 bytes) 000464[0] 1767586279.524854 write 0x987000 thru 0x98b337 (0x4338 bytes) 000465[1] 1767586279.539737 trunc from 0x98b338 to 0x901710 (0x89c27 bytes) 000466[0] 1767586279.558466 write 0x999000 thru 0x9a6bbb (0xdbbc bytes) 000467[0] 1767586279.567370 read 0x2ac000 thru 0x2af81d (0x381e bytes) 000471[0] 1767586279.587862 write 0x09a000 thru 0x09eac6 (0x4ac7 bytes) 000474[0] 1767586279.599763 read 0x388000 thru 0x38c821 (0x4822 bytes) 000478[0] 1767586279.622405 read 0x98e000 thru 0x99d32c (0xf32d bytes) 000480[0] 1767586279.635688 write 0x75b000 thru 0x75cdf3 (0x1df4 bytes) 000481[0] 1767586279.638835 mapread 0x7ba000 thru 0x7c2ffb (0x8ffc bytes) 000483[0] 1767586279.664024 mapread 0x014000 thru 0x01f9aa (0xb9ab bytes) 000484[0] 1767586279.677825 mapwrite 0x901000 thru 0x9106dc (0xf6dd bytes) 000488[1] 1767586279.732845 mapread 0x863000 thru 0x86aaed (0x7aee bytes) 000491[0] 1767586279.765871 mapread 0x5df000 thru 0x5e6a3d (0x7a3e bytes) 000492[0] 1767586279.778699 mapwrite 0x090000 thru 0x09ee80 (0xee81 bytes) 000493[1] 1767586279.834104 read_OD 0x25b000 thru 0x25cfff (0x2000 bytes) 000496[0] 1767586279.850065 trunc from 0x9a6bbc to 0x1d96ee (0x7cd4cd bytes) 000497[1] 1767586279.872058 mapwrite 0x7ed000 thru 0x7f0924 (0x3925 bytes) All operations completed A-OK! PASS 16a (19s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 23:11:22 (1767586282) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.159808 s, 52.5 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.13029 s, 64.4 MB/s Chance of close/open is 1 in 50 Seed set to 6283 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x88fa0e 000002[0] 1767586283.783959 trunc from 00000000 to 0x88fa0d (0x88fa0e bytes) 000003[1] 1767586283.811336 mapread 0x3f4a3d thru 0x3f67ba (0x1d7e bytes) 000004[1] 1767586283.838624 read 0x6abfdf thru 0x6ad83c (0x185e bytes) 000005[1] 1767586283.847128 read 0x474202 thru 0x4758b0 (0x16af bytes) 000008[1] 1767586283.856645 mapread 0x571734 thru 0x572bd9 (0x14a6 bytes) 000011[0] 1767586283.863534 read 0x2256a3 thru 0x226e58 (0x17b6 bytes) 000012[1] 1767586283.866773 mapread 0x154bc8 thru 0x154c0c (0x45 bytes) 000014[0] 1767586283.870566 write 0x30b7c0 thru 0x30c0ed (0x92e bytes) 000015[1] 1767586283.890072 mapwrite 0x018053 thru 0x018734 (0x6e2 bytes) 000018[0] 1767586283.944383 mapwrite 0x88a5cc thru 0x88bca0 (0x16d5 bytes) 000022[1] 1767586283.989284 mapread 0x35907a thru 0x359924 (0x8ab bytes) 000023[0] 1767586284.003237 mapwrite 0x1ff3fe thru 0x2007b5 (0x13b8 bytes) 000024[0] 1767586284.041821 trunc from 0x88fa0e to 0x5766f3 (0x31931a bytes) 000027[0] 1767586284.067904 mapwrite 0x43ba3d thru 0x43c276 (0x83a bytes) 000029[1] 1767586284.130753 trunc from 0x5766f4 to 0x089aed (0x4ecc06 bytes) 000033[1] 1767586284.178201 mapread 0x02869c thru 0x0295a9 (0xf0e bytes) 000034[1] 1767586284.193232 read 0x06fa79 thru 0x06fc86 (0x20e bytes) 000035[0] 1767586284.205653 mapread 0x0851ec thru 0x087061 (0x1e76 bytes) 000038[0] 1767586284.245587 read 0x003452 thru 0x003fc2 (0xb71 bytes) 000042[0] 1767586284.273219 write 0x1e253a thru 0x1e2d34 (0x7fb bytes) 000046[0] 1767586284.313349 write 0x09855f thru 0x098b29 (0x5cb bytes) 000047[0] 1767586284.338999 mapread 0x13e15c thru 0x13fbd5 (0x1a7a bytes) 000048[0] 1767586284.351367 trunc from 0x1e2d35 to 0x05a4a0 (0x188894 bytes) 000050[0] 1767586284.384664 mapread 0x02b895 thru 0x02c208 (0x974 bytes) 000054[1] 1767586284.423712 read 0x048ce2 thru 0x04a555 (0x1874 bytes) 000056[0] 1767586284.448078 trunc from 0x05a4a1 to 0x249122 (0x1eec82 bytes) 000057[0] 1767586284.462630 read 0x1682d1 thru 0x169d0f (0x1a3f bytes) 000058[0] 1767586284.500816 mapwrite 0x784bb2 thru 0x7863bb (0x180a bytes) 000060[0] 1767586284.602478 mapwrite 0x05b695 thru 0x05ceb8 (0x1824 bytes) 000062[1] 1767586284.641120 write 0x976f6a thru 0x97848c (0x1523 bytes) 000063[1] 1767586284.662238 mapread 0x8c2581 thru 0x8c3ca1 (0x1721 bytes) truncating to largest ever: 0x954a18 000066[1] 1767586284.751006 trunc from 0x97848d to 0x954a17 (0x23a75 bytes) 000067[1] 1767586284.772141 mapread 0x758f94 thru 0x75912b (0x198 bytes) 000068[1] 1767586284.841076 trunc from 0x954a18 to 0x7780c9 (0x1dc94e bytes) 000070[1] 1767586284.863502 mapread 0x74ef08 thru 0x7508d4 (0x19cd bytes) 000074[0] 1767586284.883100 mapwrite 0x6351f3 thru 0x636752 (0x1560 bytes) 000077[0] 1767586284.951427 mapread 0x5cd767 thru 0x5ce66a (0xf04 bytes) 000079[1] 1767586284.977457 trunc from 0x7780ca to 0x8dcd44 (0x164c7b bytes) truncating to largest ever: 0x98ab5d 000080[1] 1767586285.001120 trunc from 0x8dcd45 to 0x98ab5c (0xade18 bytes) 000081[0] 1767586285.011345 trunc from 0x98ab5d to 0x8b24b7 (0xd86a5 bytes) 000082[1] 1767586285.045730 mapread 0x74225c thru 0x742968 (0x70d bytes) 000083[0] 1767586285.061954 read 0x74e6d0 thru 0x74eef8 (0x829 bytes) 000084[0] 1767586285.065544 read 0x1e89c2 thru 0x1e9907 (0xf46 bytes) 000088[1] 1767586285.079046 write 0x3b5c72 thru 0x3b6810 (0xb9f bytes) 000091[1] 1767586285.106626 read 0x784a35 thru 0x78601b (0x15e7 bytes) 000092[1] 1767586285.110313 write 0x24b9ea thru 0x24bffd (0x614 bytes) 000094[0] 1767586285.129940 write 0x40b0bd thru 0x40c7e5 (0x1729 bytes) 000095[0] 1767586285.147845 mapread 0x1cd255 thru 0x1cda60 (0x80c bytes) 000100[0] 1767586285.190679 write 0x77e515 thru 0x77ee99 (0x985 bytes) 000101[1] 1767586285.193738 trunc from 0x8b24b8 to 0x13cc50 (0x775867 bytes) 000103[0] 1767586285.228015 write 0x928708 thru 0x928770 (0x69 bytes) 000106[1] 1767586285.263148 mapwrite 0x569290 thru 0x569d81 (0xaf2 bytes) 000108[0] 1767586285.318119 mapwrite 0x65b0f0 thru 0x65b7ea (0x6fb bytes) 000110[1] 1767586285.359837 read 0x512800 thru 0x512faf (0x7b0 bytes) 000111[1] 1767586285.369789 mapwrite 0x3ca6c6 thru 0x3cb3e1 (0xd1c bytes) 000124[0] 1767586285.461446 mapwrite 0x58405a thru 0x58430c (0x2b3 bytes) 000125[1] 1767586285.501828 read 0x04a0ea thru 0x04bf42 (0x1e59 bytes) 000127[0] 1767586285.522001 mapread 0x6905d4 thru 0x6906c4 (0xf1 bytes) 000129[1] 1767586285.549959 read 0x388598 thru 0x389bcf (0x1638 bytes) 000130[0] 1767586285.561787 trunc from 0x928771 to 0x1562b3 (0x7d24bd bytes) 000131[0] 1767586285.584800 mapwrite 0x74db79 thru 0x74f1c5 (0x164d bytes) 000133[0] 1767586285.624273 mapwrite 0x5d4bbd thru 0x5d68bc (0x1d00 bytes) 000134[0] 1767586285.673012 write 0x744006 thru 0x74402f (0x2a bytes) 000136[1] 1767586285.692340 trunc from 0x74f1c6 to 0x5a07d5 (0x1ae9f0 bytes) 000138[0] 1767586285.727824 trunc from 0x5a07d6 to 0x2ba882 (0x2e5f53 bytes) 000139[1] 1767586285.756842 read 0x2adfda thru 0x2af67a (0x16a1 bytes) 000140[1] 1767586285.807190 mapread 0x08275c thru 0x0841a0 (0x1a45 bytes) 000141[1] 1767586285.817180 write 0x358efa thru 0x35971b (0x822 bytes) 000143[1] 1767586285.844957 mapread 0x1c24a0 thru 0x1c361a (0x117b bytes) 000144[1] 1767586285.869063 trunc from 0x35971c to 0x5bb1ec (0x261ad1 bytes) 000145[0] 1767586285.894019 write 0x1c1a58 thru 0x1c300f (0x15b8 bytes) 000146[1] 1767586285.919171 trunc from 0x5bb1ed to 0x622f80 (0x67d94 bytes) 000152[0] 1767586285.968856 read 0x3b0b4b thru 0x3b192b (0xde1 bytes) 000155[1] 1767586285.995960 trunc from 0x622f81 to 0x6edaa7 (0xcab27 bytes) 000156[1] 1767586286.018817 mapread 0x070105 thru 0x07148c (0x1388 bytes) 000158[1] 1767586286.044616 mapread 0x66a7c7 thru 0x66b5d3 (0xe0d bytes) 000159[1] 1767586286.092810 mapwrite 0x5dd18e thru 0x5dd7a7 (0x61a bytes) 000162[0] 1767586286.131986 read 0x1a92c1 thru 0x1aa528 (0x1268 bytes) 000166[0] 1767586286.169402 trunc from 0x6edaa8 to 0x0c0870 (0x62d237 bytes) 000167[1] 1767586286.202855 mapwrite 0x7f5f73 thru 0x7f70f1 (0x117f bytes) 000169[1] 1767586286.270087 trunc from 0x7f70f2 to 0x84f863 (0x58772 bytes) 000172[0] 1767586286.309014 write 0x7da122 thru 0x7dae79 (0xd58 bytes) 000174[1] 1767586286.338645 write 0x7a7ce6 thru 0x7a873a (0xa55 bytes) 000178[1] 1767586286.389015 read 0x7b45ea thru 0x7b5bb0 (0x15c7 bytes) 000179[0] 1767586286.401238 trunc from 0x84f864 to 0x6b39d3 (0x19be90 bytes) 000180[0] 1767586286.432017 mapread 0x438568 thru 0x439e5b (0x18f4 bytes) 000181[0] 1767586286.446643 trunc from 0x6b39d4 to 0x57a639 (0x13939a bytes) 000183[0] 1767586286.468183 mapread 0x354641 thru 0x3550be (0xa7e bytes) 000188[1] 1767586286.509736 mapwrite 0x5e5700 thru 0x5e5861 (0x162 bytes) 000190[0] 1767586286.570227 mapwrite 0x596569 thru 0x596da4 (0x83c bytes) 000191[0] 1767586286.609422 trunc from 0x5e5862 to 0x988039 (0x3a27d8 bytes) 000194[0] 1767586286.650801 trunc from 0x98803a to 0x1f529e (0x792d9b bytes) 000197[0] 1767586286.686969 write 0x2df134 thru 0x2df88d (0x75a bytes) 000199[1] 1767586286.706795 mapread 0x215ac5 thru 0x2172a4 (0x17e0 bytes) 000204[1] 1767586286.760765 mapwrite 0x2f8244 thru 0x2f9752 (0x150f bytes) 000207[1] 1767586286.825283 mapwrite 0x038c57 thru 0x03a344 (0x16ee bytes) 000209[0] 1767586286.875082 write 0x8ae9db thru 0x8b02fb (0x1921 bytes) 000212[1] 1767586286.908993 mapwrite 0x159699 thru 0x15a089 (0x9f1 bytes) 000213[1] 1767586286.947819 trunc from 0x8b02fc to 0x983948 (0xd364d bytes) 000214[1] 1767586286.967014 read 0x964f15 thru 0x966311 (0x13fd bytes) 000215[0] 1767586286.979049 mapread 0x7a436f thru 0x7a6330 (0x1fc2 bytes) 000218[0] 1767586287.019890 read 0x900f7a thru 0x902094 (0x111b bytes) 000221[0] 1767586287.035460 write 0x033a8c thru 0x0348a6 (0xe1b bytes) 000222[1] 1767586287.054860 mapwrite 0x5f964b thru 0x5fa037 (0x9ed bytes) 000223[0] 1767586287.096011 mapwrite 0x4c5c7b thru 0x4c6c0e (0xf94 bytes) 000224[0] 1767586287.134286 trunc from 0x983949 to 0x256632 (0x72d316 bytes) 000225[1] 1767586287.145187 read 0x24307a thru 0x244ac2 (0x1a49 bytes) 000226[0] 1767586287.166098 mapwrite 0x1c2154 thru 0x1c3dba (0x1c67 bytes) 000227[1] 1767586287.208024 mapread 0x108a78 thru 0x1091af (0x738 bytes) 000230[1] 1767586287.234679 mapwrite 0x04f80a thru 0x05096f (0x1166 bytes) 000236[1] 1767586287.316869 mapread 0x10b09e thru 0x10c3d1 (0x1334 bytes) 000237[1] 1767586287.333988 read 0x1a36f5 thru 0x1a5087 (0x1993 bytes) 000238[1] 1767586287.377384 trunc from 0x256633 to 0x984d47 (0x72e715 bytes) 000240[1] 1767586287.394075 mapread 0x076927 thru 0x078521 (0x1bfb bytes) 000241[1] 1767586287.394901 mapwrite 0x0d3d73 thru 0x0d42b7 (0x545 bytes) 000242[0] 1767586287.415599 read 0x0256e1 thru 0x0272c2 (0x1be2 bytes) 000246[1] 1767586287.497486 mapwrite 0x73a938 thru 0x73b8c2 (0xf8b bytes) 000247[0] 1767586287.586449 read 0x0a37f2 thru 0x0a49d6 (0x11e5 bytes) 000251[0] 1767586287.596862 trunc from 0x984d48 to 0x4d6213 (0x4aeb34 bytes) 000254[0] 1767586287.657837 read 0x244bb3 thru 0x245b88 (0xfd6 bytes) 000260[1] 1767586287.690546 mapread 0x3a1fe9 thru 0x3a37b2 (0x17ca bytes) 000263[1] 1767586287.717971 trunc from 0x4d6214 to 0x3d5766 (0x100aad bytes) 000265[1] 1767586287.737031 read 0x0881f3 thru 0x08a0a4 (0x1eb2 bytes) 000266[0] 1767586287.788788 trunc from 0x3d5767 to 0x639fa6 (0x264840 bytes) 000267[0] 1767586287.814886 write 0x2548c8 thru 0x2563d1 (0x1b0a bytes) 000268[0] 1767586287.820406 mapwrite 0x4ec363 thru 0x4ed9e2 (0x1680 bytes) 000269[0] 1767586287.844384 mapread 0x1baf26 thru 0x1bbab7 (0xb92 bytes) 000270[0] 1767586287.850493 trunc from 0x639fa7 to 0x41e907 (0x21b69f bytes) 000276[0] 1767586287.900633 write 0x2c1f43 thru 0x2c3171 (0x122f bytes) 000280[0] 1767586287.917492 read 0x09b07a thru 0x09b603 (0x58a bytes) 000282[1] 1767586287.921072 trunc from 0x41e908 to 0x17aae7 (0x2a3e20 bytes) 000283[1] 1767586287.943581 write 0x813223 thru 0x814bdd (0x19bb bytes) 000285[0] 1767586287.958886 trunc from 0x814bde to 0x13c4b4 (0x6d8729 bytes) 000286[0] 1767586287.981328 mapwrite 0x12cb59 thru 0x12dca5 (0x114d bytes) 000287[1] 1767586288.008058 mapwrite 0x5bbdee thru 0x5bc8ab (0xabe bytes) 000289[0] 1767586288.075648 read 0x3f671c thru 0x3f8450 (0x1d35 bytes) 000290[1] 1767586288.139393 mapwrite 0x132b5a thru 0x134780 (0x1c27 bytes) 000291[1] 1767586288.193798 trunc from 0x5bc8ac to 0x4d3e26 (0xe8a85 bytes) 000292[1] 1767586288.215034 read 0x380d18 thru 0x382c5b (0x1f44 bytes) 000293[0] 1767586288.221067 mapread 0x416ea8 thru 0x41793e (0xa97 bytes) 000297[1] 1767586288.266145 write 0x2233a9 thru 0x224ebe (0x1b16 bytes) 000298[1] 1767586288.272216 write 0x4eff6a thru 0x4f0f7e (0x1015 bytes) 000299[1] 1767586288.279686 write 0x373152 thru 0x3737b8 (0x667 bytes) 000300[0] 1767586288.282715 mapwrite 0x37a0f6 thru 0x37b85f (0x176a bytes) 000301[1] 1767586288.329800 read 0x0a2a6a thru 0x0a4236 (0x17cd bytes) 000302[0] 1767586288.346085 write 0x42dccf thru 0x42e5ca (0x8fc bytes) 000303[0] 1767586288.369736 trunc from 0x4f0f7f to 0x198460 (0x358b1e bytes) 000305[1] 1767586288.385698 trunc from 0x198461 to 0x5cd549 (0x4350e9 bytes) 000310[1] 1767586288.437151 mapread 0x390d54 thru 0x391dca (0x1077 bytes) 000311[1] 1767586288.443449 read 0x2dfa90 thru 0x2dfb1c (0x8d bytes) 000312[0] 1767586288.454997 mapread 0x59f52d thru 0x5a0570 (0x1044 bytes) 000313[1] 1767586288.481330 read 0x5c2efa thru 0x5c3dff (0xf06 bytes) 000317[0] 1767586288.486170 mapread 0x2d68a6 thru 0x2d74a7 (0xc02 bytes) 000318[0] 1767586288.501262 mapwrite 0x0e8164 thru 0x0e8e7a (0xd17 bytes) 000319[0] 1767586288.531210 mapread 0x019064 thru 0x01aec9 (0x1e66 bytes) 000320[1] 1767586288.546048 read 0x0c4757 thru 0x0c5b33 (0x13dd bytes) 000321[0] 1767586288.556048 mapwrite 0x0cb175 thru 0x0cc5dc (0x1468 bytes) 000322[0] 1767586288.587186 write 0x621b66 thru 0x6225e0 (0xa7b bytes) 000324[0] 1767586288.595678 trunc from 0x6225e1 to 0x8a4e06 (0x282826 bytes) 000325[1] 1767586288.616526 mapwrite 0x485611 thru 0x485d9a (0x78a bytes) 000329[1] 1767586288.689783 mapread 0x17e38d thru 0x17ec29 (0x89d bytes) 000330[0] 1767586288.709777 trunc from 0x8a4e07 to 0x09b195 (0x809c71 bytes) 000331[1] 1767586288.737978 read 0x09a882 thru 0x09b195 (0x914 bytes) 000332[0] 1767586288.761981 mapread 0x04588c thru 0x0468cb (0x1040 bytes) 000334[1] 1767586288.787461 read 0x035d55 thru 0x0376ad (0x1959 bytes) 000335[1] 1767586288.796041 trunc from 0x09b196 to 0x3091b8 (0x26e023 bytes) 000336[0] 1767586288.813299 mapread 0x0433b0 thru 0x043f04 (0xb55 bytes) 000337[0] 1767586288.835515 mapwrite 0x6b878f thru 0x6b9c81 (0x14f3 bytes) 000339[0] 1767586288.899695 write 0x348095 thru 0x34915d (0x10c9 bytes) 000341[0] 1767586288.921884 read 0x4dd3df thru 0x4de68c (0x12ae bytes) 000343[0] 1767586288.934884 mapwrite 0x0319e1 thru 0x031b13 (0x133 bytes) 000347[1] 1767586288.969625 write 0x453d43 thru 0x455b72 (0x1e30 bytes) 000349[0] 1767586288.997341 mapread 0x116a9c thru 0x116ce3 (0x248 bytes) 000350[1] 1767586289.012843 mapread 0x69928b thru 0x69b1e0 (0x1f56 bytes) 000352[0] 1767586289.042703 mapwrite 0x55bd7c thru 0x55d599 (0x181e bytes) 000353[0] 1767586289.092975 read 0x3924f3 thru 0x394015 (0x1b23 bytes) 000354[0] 1767586289.106188 mapwrite 0x02b243 thru 0x02bc0a (0x9c8 bytes) 000355[1] 1767586289.130201 mapwrite 0x8c2195 thru 0x8c2a7e (0x8ea bytes) 000356[0] 1767586289.226753 write 0x7a737e thru 0x7a7aaa (0x72d bytes) 000357[0] 1767586289.244588 mapread 0x278742 thru 0x2790df (0x99e bytes) 000367[1] 1767586289.310105 write 0x089737 thru 0x08a6a6 (0xf70 bytes) 000368[1] 1767586289.328640 read 0x16c317 thru 0x16d93f (0x1629 bytes) 000369[1] 1767586289.339874 write 0x64b42a thru 0x64c2c3 (0xe9a bytes) 000374[1] 1767586289.390238 read 0x885899 thru 0x88737d (0x1ae5 bytes) 000375[1] 1767586289.402715 trunc from 0x8c2a7f to 0x3dedc8 (0x4e3cb6 bytes) 000381[0] 1767586289.435840 mapread 0x000f53 thru 0x0029a5 (0x1a53 bytes) 000383[0] 1767586289.466813 mapwrite 0x1b9f82 thru 0x1bbe72 (0x1ef1 bytes) 000385[1] 1767586289.524714 write 0x33e61f thru 0x33fa1f (0x1401 bytes) 000386[1] 1767586289.535280 read 0x028e56 thru 0x028f7a (0x125 bytes) 000387[0] 1767586289.586039 read 0x28bd36 thru 0x28c03a (0x305 bytes) 000388[0] 1767586289.654346 mapread 0x262960 thru 0x26323e (0x8df bytes) 000395[0] 1767586289.702590 mapwrite 0x68660b thru 0x687137 (0xb2d bytes) 000397[0] 1767586289.801271 mapwrite 0x702d0e thru 0x704741 (0x1a34 bytes) 000398[0] 1767586289.841397 read 0x608ba4 thru 0x609fc6 (0x1423 bytes) 000401[1] 1767586289.854398 read 0x5636b6 thru 0x5655bb (0x1f06 bytes) 000414[0] 1767586289.940873 mapwrite 0x840e7f thru 0x841faf (0x1131 bytes) 000417[0] 1767586290.010738 mapwrite 0x978324 thru 0x9794c4 (0x11a1 bytes) 000420[1] 1767586290.068863 trunc from 0x9794c5 to 0x8c4119 (0xb53ab bytes) 000421[0] 1767586290.095834 mapread 0x0364c2 thru 0x036627 (0x166 bytes) 000423[0] 1767586290.120865 read 0x5be779 thru 0x5c0439 (0x1cc1 bytes) 000425[0] 1767586290.132231 read 0x2eeae5 thru 0x2ef0a1 (0x5bd bytes) 000426[1] 1767586290.135630 trunc from 0x8c411a to 0x103787 (0x7c0992 bytes) 000428[0] 1767586290.167174 read 0x04f166 thru 0x04f903 (0x79e bytes) 000430[0] 1767586290.189575 trunc from 0x103788 to 0x121d6a (0x1e5e3 bytes) 000433[0] 1767586290.211345 write 0x9d7f66 thru 0x9d9aeb (0x1b86 bytes) 000434[0] 1767586290.215290 mapread 0x594972 thru 0x5966df (0x1d6e bytes) 000435[0] 1767586290.286558 write 0x2bd6be thru 0x2be91c (0x125f bytes) 000436[1] 1767586290.287471 mapwrite 0x813e7f thru 0x815ae3 (0x1c65 bytes) 000437[0] 1767586290.348812 trunc from 0x9d9aec to 0x498df2 (0x540cf9 bytes) 000447[1] 1767586290.417423 trunc from 0x498df3 to 0x718774 (0x27f982 bytes) 000448[0] 1767586290.436830 write 0x14e417 thru 0x14eee7 (0xad1 bytes) 000450[1] 1767586290.463760 mapread 0x1fe28a thru 0x1fe9ff (0x776 bytes) 000453[0] 1767586290.488327 read 0x100b48 thru 0x1021e7 (0x16a0 bytes) 000454[0] 1767586290.491573 trunc from 0x718775 to 0x8ab16e (0x1929fa bytes) 000457[0] 1767586290.521687 trunc from 0x8ab16f to 0x85b1a9 (0x4ffc5 bytes) 000462[0] 1767586290.569871 write 0x749584 thru 0x74b032 (0x1aaf bytes) 000463[1] 1767586290.575434 mapread 0x6df55a thru 0x6e1235 (0x1cdc bytes) 000464[1] 1767586290.604362 trunc from 0x85b1aa to 0x22be78 (0x62f331 bytes) 000465[1] 1767586290.631773 write 0x35efcf thru 0x360d01 (0x1d33 bytes) 000469[1] 1767586290.654406 read 0x35317f thru 0x3546d6 (0x1558 bytes) 000470[0] 1767586290.714536 mapwrite 0x0ea7c8 thru 0x0eb8e0 (0x1119 bytes) 000471[0] 1767586290.767811 mapwrite 0x26e43c thru 0x26f594 (0x1159 bytes) 000473[0] 1767586290.816803 mapread 0x026e0d thru 0x0280d2 (0x12c6 bytes) 000475[1] 1767586290.847845 trunc from 0x360d02 to 0x7659fe (0x404cfd bytes) 000477[0] 1767586290.865403 mapread 0x2dbef0 thru 0x2dd977 (0x1a88 bytes) 000482[1] 1767586290.917402 write 0x0d8f63 thru 0x0d9dea (0xe88 bytes) 000483[0] 1767586290.944986 mapwrite 0x4f0568 thru 0x4f23b9 (0x1e52 bytes) 000490[0] 1767586291.045809 trunc from 0x7659ff to 0x3b93c7 (0x3ac637 bytes) 000491[0] 1767586291.065197 mapread 0x383c32 thru 0x383f2d (0x2fc bytes) 000493[1] 1767586291.089854 trunc from 0x3b93c8 to 0x37c2c7 (0x3d100 bytes) 000494[0] 1767586291.108376 mapread 0x3303c5 thru 0x3311bc (0xdf8 bytes) 000498[1] 1767586291.156897 trunc from 0x37c2c8 to 0x765084 (0x3e8dbd bytes) 000499[0] 1767586291.172217 read 0x2f6a79 thru 0x2f79b8 (0xf40 bytes) 000500[1] 1767586291.239418 mapread 0x155d26 thru 0x156d9b (0x1076 bytes) All operations completed A-OK! PASS 16b (11s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 23:11:33 (1767586293) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (2s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 23:11:35 (1767586295) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.171825 s, 48.8 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.137606 s, 61.0 MB/s ldlm.namespaces.MGC192.168.204.154@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff993903017800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff9939106e5800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff993903017800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff9939106e5800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff993903017800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff9939106e5800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.06494 s, 25.8 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.16401 s, 90.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.705163 s, 149 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.33694 s, 44.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.86361 s, 36.6 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.25001 s, 83.9 MB/s PASS 16d (18s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 23:11:53 (1767586313) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.102296 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0521708 s, 20.1 MB/s PASS 16e (3s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 23:11:56 (1767586316) TIMEOUT PASS 16f (23s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 23:12:19 (1767586339) TIMEOUT PASS 16g (23s) == sanityn test 16h: mmap read after truncate file ======= 23:12:42 (1767586362) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16h (4s) == sanityn test 16i: read after truncate file ============ 23:12:46 (1767586366) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (3s) == sanityn test 16j: race dio with buffered i/o ========== 23:12:49 (1767586369) 1+0 records in 1+0 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.752464 s, 34.8 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.533871 s, 49.1 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.877108 s, 29.9 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.521561 s, 50.3 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.70512 s, 37.2 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.595809 s, 44.0 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.813694 s, 32.2 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.487507 s, 53.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.588265 s, 44.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.907363 s, 28.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.986767 s, 26.6 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.544352 s, 48.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.619213 s, 42.3 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.730275 s, 35.9 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.801148 s, 32.7 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.613325 s, 42.7 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.694122 s, 37.8 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.793626 s, 33.0 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.827035 s, 31.7 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK PASS 16j (13s) == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 23:13:02 (1767586382) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x1a7062c skipping zero size read truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x32cce75 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x32cce75 truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 All operations completed A-OK! Sun Jan 4 23:13:12 EST 2026 + fsx 2 done: rc=0 Sun Jan 4 23:13:12 EST 2026 + fsx 3 done: rc=0 Sun Jan 4 23:13:12 EST 2026 + fsx 4 done: rc=0 Sun Jan 4 23:13:12 EST 2026 + fsx 5 done: rc=0 Sun Jan 4 23:13:12 EST 2026 PASS 16k (13s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 23:13:15 (1767586395) fail_loc=0x8000030a PASS 17 (6s) == sanityn test 18: mmap sanity check =========================================================================================== 23:13:21 (1767586401) mmap test1: basic mmap operation (PASS, 0.045265s) mmap test2: MAP_PRIVATE not write back (PASS, 0.032749s) mmap test3: concurrent mmap ops on two nodes (PASS, 56.619s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 4.1831s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.064345s) mmap test6: check mmap write/read content on two nodes (PASS, 0.07618s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: multi-thread mmap access (PASS, 0.40425s) mmap test11: multi-thread shared mmap (PASS, 0.37163s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0393s) PASS 18 (67s) == sanityn test 19: test concurrent uncached read races ========================================================================= 23:14:28 (1767586468) oleg454-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg454-client: oleg454-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (2s) == sanityn test 20: test extra readahead page left in cache ============================================================== 23:14:30 (1767586470) PASS 20 (3s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 23:14:33 (1767586473) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (3s) == sanityn test 23: others should see updated atime while another read============================================================== 23:14:36 (1767586476) now is 1767586477 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 new atime is 1767586538 PASS 23 (65s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 23:15:41 (1767586541) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 388 490.2K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 400 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 114.5K 401 114.2K 1% /mnt/lustre[OST:1] filesystem_summary: 229.4K 388 229.0K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 388 490.2K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 400 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 401 114.2K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.4K 388 229.0K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 8.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 30.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 38.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502379 388 501991 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118000 400 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117297 401 116896 1% /mnt/lustre2[OST:1] filesystem_summary: 234884 388 234496 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 388 490.2K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 400 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.5K 401 114.2K 1% /mnt/lustre2[OST:1] filesystem_summary: 229.4K 388 229.0K 1% /mnt/lustre2 Reader error: 'No devices found' at 0 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502379 388 501991 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118000 400 117600 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 117297 401 116896 1% /mnt/lustre[OST:1] filesystem_summary: 234884 388 234496 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502379 388 501991 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118000 400 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 117297 401 116896 1% /mnt/lustre2[OST:1] filesystem_summary: 234884 388 234496 1% /mnt/lustre2 Reader error: 'No devices found' at 0 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 30720 3738624 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 38912 7499776 1% /mnt/lustre2 PASS 24a (3s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 23:15:44 (1767586544) PASS 24b (3s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 23:15:47 (1767586547) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (3s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 23:15:50 (1767586550) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 23:15:51 (1767586551) utime: good mknod times 1767586551 <= 1767586552 <= 1767586552 for /mnt/lustre/f26a utime: good mknod times 1767586551 <= 1767586552 <= 1767586552 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (5s) == sanityn test 26b: sync mtime between ost and mds ====== 23:15:56 (1767586556) PASS 26b (4s) == sanityn test 26c: set-in-past on open file is not lost on close ========================================================== 23:16:01 (1767586561) multiop /mnt/lustre/f26c.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 File: /mnt/lustre/f26c.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2026-01-04 23:16:01.000000000 -0500 Birth: 2026-01-04 23:16:02.000000000 -0500 File: /mnt/lustre2/f26c.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502398 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2026-01-04 23:16:01.000000000 -0500 Birth: 2026-01-04 23:16:02.000000000 -0500 PASS 26c (3s) SKIP: sanityn test_27 skipping excluded test 27 == sanityn test 30: recreate file race =================== 23:16:05 (1767586565) 0 PASS 30 (5s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 23:16:10 (1767586570) fail_loc=0x314 PASS 31a (4s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 23:16:14 (1767586574) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg454-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.204.154@tcp:/lustre 7542784 39936 7496704 1% /mnt/lustre2 PASS 31b (13s) == sanityn test 31r: open-rename(replace) race =========== 23:16:28 (1767586588) -rw-r--r-- 1 root root 0 Jan 4 23:16 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (6s) == sanityn test 31s: open should not revalidate invalid dentry ========================================================== 23:16:34 (1767586594) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 1207: 42830 Terminated tail -f $tfile PASS 31s (3s) == sanityn test 31t: getattr should not revalidate invalid dentry ========================================================== 23:16:37 (1767586597) PASS 31t (4s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 23:16:42 (1767586602) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (2s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 23:16:44 (1767586604) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: dependent transactions should trigger COS ========================================================== 23:16:45 (1767586605) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (2s) == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 23:16:47 (1767586607) SKIP: sanityn test_33e needs >= 2 MDTs SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 23:16:48 (1767586608) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg454-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff993903017800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff993903017800.ost_server_uuid in IDLE state after 0 sec oleg454-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff993903017800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff993903017800.ost_server_uuid in FULL state after 0 sec oleg454-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff993903017800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff993903017800.ost_server_uuid in IDLE state after 0 sec oleg454-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff993903017800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff993903017800.ost_server_uuid in FULL state after 0 sec oleg454-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff993903017800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff993903017800.ost_server_uuid in IDLE state after 0 sec oleg454-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff993903017800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff993903017800.ost_server_uuid in IDLE state after 0 sec PASS 34 (99s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 23:18:27 (1767586707) fail_loc=0x80000317 Race attempt 0 Wait for 48953 49072 for 60 sec... - open/close 2127 (time 1767586720.72 total 10.00 last 212.60) total: 4000 open/close in 19.04 seconds: 210.03 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.204.154@tcp:/lustre 7.2G 41M 7.2G 1% /mnt/lustre 192.168.204.154@tcp:/lustre 7.2G 41M 7.2G 1% /mnt/lustre2 PASS 35 (66s) == sanityn test 36: handle ESTALE/open-unlink correctly == 23:19:33 (1767586773) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg454-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.373071 s, 141 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7496704) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.56424 s, 92.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7496704) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.39959 s, 131 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7494656) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.496151 s, 106 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7496704) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.478096 s, 110 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7496704) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.46107 s, 114 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7496704) after_dd(7443456) after(7494656) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.369929 s, 142 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7494656) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.366121 s, 143 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7496704) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.343188 s, 153 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7496704) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.408194 s, 128 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7496704) after_dd(7445504) after(7496704) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.45234 s, 116 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7496704) after_dd(7441408) after(7492608) PASS 36 (236s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 23:23:29 (1767587009) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 - create 4998 (time 1767587021.68 total 10.00 last 499.76) - create 9969 (time 1767587031.68 total 20.00 last 497.02) total: 10000 create in 20.07 seconds: 498.34 ops/second PASS 37 (25s) == sanityn test 39a: file mtime does not change after rename ========================================================== 23:23:54 (1767587034) repeat after cancel_lru_locks PASS 39a (4s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 23:23:58 (1767587038) repeat after cancel_lru_locks PASS 39b (4s) == sanityn test 39c: check truncate mtime update ================================================================================ 23:24:02 (1767587042) repeat after cancel_lru_locks PASS 39c (4s) == sanityn test 39d: sync write should update mtime ====== 23:24:06 (1767587046) fail_loc=0x411 fail_loc=0 PASS 39d (3s) SKIP: sanityn test_40a skipping ALWAYS excluded test 40a == sanityn test 40b: pdirops: open|create and others ======================================================================== 23:24:10 (1767587050) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (10s) == sanityn test 40c: pdirops: link and others ======================================================================== 23:24:20 (1767587060) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (10s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 23:24:30 (1767587070) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (10s) == sanityn test 40e: pdirops: rename and others ======================================================================== 23:24:40 (1767587080) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (10s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 23:24:50 (1767587090) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (7s) == sanityn test 41b: pdirops: create vs create ======================================================================== 23:24:57 (1767587097) open: File exists Conflict PASS 41b (6s) == sanityn test 41c: pdirops: create vs link ======================================================================== 23:25:03 (1767587103) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (7s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 23:25:10 (1767587110) Conflict PASS 41d (7s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 23:25:17 (1767587117) Conflict PASS 41e (7s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 23:25:24 (1767587124) Conflict PASS 41f (7s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 23:25:31 (1767587131) Conflict PASS 41g (7s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 23:25:38 (1767587138) Conflict PASS 41h (7s) == sanityn test 41i: reint_open: create vs create ======== 23:25:45 (1767587145) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (811s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 23:39:16 (1767587956) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (7s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 23:39:23 (1767587963) open: File exists Conflict PASS 42b (7s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 23:39:30 (1767587970) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (7s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 23:39:37 (1767587977) Conflict PASS 42d (7s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 23:39:44 (1767587984) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (7s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 23:39:51 (1767587991) Conflict PASS 42f (7s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 23:39:58 (1767587998) Conflict PASS 42g (7s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 23:40:05 (1767588005) Conflict PASS 42h (7s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 23:40:12 (1767588012) PASS 43a (27s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 23:40:39 (1767588039) Conflict PASS 43b (7s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 23:40:46 (1767588046) Conflict PASS 43c (7s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 23:40:53 (1767588053) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (7s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 23:41:00 (1767588060) Conflict PASS 43e (7s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 23:41:07 (1767588067) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (7s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 23:41:14 (1767588074) stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (7s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 23:41:21 (1767588081) Conflict PASS 43h (7s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 23:41:28 (1767588088) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 23:41:29 (1767588089) mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (66s) == sanityn test 43k: unlink vs create ==================== 23:42:35 (1767588155) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (635s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 23:53:10 (1767588790) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Sun Jan 4 23:53:15 EST 2026 PASS 44a (7s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 23:53:17 (1767588797) open: File exists Conflict PASS 44b (7s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 23:53:24 (1767588804) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (7s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 23:53:31 (1767588811) Conflict PASS 44d (7s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 23:53:38 (1767588818) Conflict PASS 44e (7s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 23:53:45 (1767588825) Conflict PASS 44f (7s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 23:53:52 (1767588832) Conflict PASS 44g (7s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 23:53:59 (1767588839) Conflict PASS 44h (7s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 23:54:06 (1767588846) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 23:54:07 (1767588847) PASS 45a (34s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 23:54:41 (1767588881) Conflict PASS 45b (7s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 23:54:48 (1767588888) Conflict PASS 45c (6s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 23:54:54 (1767588894) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (7s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 23:55:01 (1767588901) Conflict PASS 45e (6s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 23:55:07 (1767588907) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (7s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 23:55:14 (1767588914) stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (7s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 23:55:21 (1767588921) Conflict PASS 45h (7s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 23:55:28 (1767588928) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 23:55:29 (1767588929) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (629s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 00:05:58 (1767589558) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (7s) == sanityn test 46b: pdirops: link vs create ======================================================================== 00:06:05 (1767589565) open: File exists Conflict PASS 46b (7s) == sanityn test 46c: pdirops: link vs link ======================================================================== 00:06:12 (1767589572) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (7s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 00:06:19 (1767589579) Conflict PASS 46d (7s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 00:06:26 (1767589586) Conflict PASS 46e (7s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 00:06:33 (1767589593) Conflict PASS 46f (7s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 00:06:40 (1767589600) Conflict PASS 46g (7s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 00:06:47 (1767589607) Conflict PASS 46h (7s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 00:06:54 (1767589614) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 00:06:55 (1767589615) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (2s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 00:06:57 (1767589617) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 00:06:58 (1767589618) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (2s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 00:07:00 (1767589620) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 00:07:01 (1767589621) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (2s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 00:07:03 (1767589623) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 00:07:04 (1767589624) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (2s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 00:07:06 (1767589626) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.00888356 s, 1.2 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (8s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 00:07:14 (1767589634) 0+1 records in 0+1 records out 158 bytes copied, 0.00667329 s, 23.7 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3510: kill: (259928) - No such process PASS 51a (5s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 00:07:19 (1767589639) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00409542 s, 250 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273456 s, 38.3 MB/s 3145728 PASS 51b (15s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 00:07:34 (1767589654) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00753059 s, 136 kB/s PASS 51c (8s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 00:07:42 (1767589662) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164862 s, 63.6 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (4s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 00:07:46 (1767589666) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2454 0x996 0x240000400 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2454 0x996 0x240000400 read lease(1) has applied. PASS 51e (5s) == sanityn test 54: rename locking ======================= 00:07:51 (1767589671) llite.lustre-ffff993903017800.xattr_cache=1 llite.lustre-ffff9939106e5800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505011 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-05 00:07:52.000000000 -0500 Modify: 2026-01-05 00:07:52.000000000 -0500 Change: 2026-01-05 00:07:52.000000000 -0500 Birth: 2026-01-05 00:07:53.000000000 -0500 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505014 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-05 00:07:58.000000000 -0500 Modify: 2026-01-05 00:07:58.000000000 -0500 Change: 2026-01-05 00:07:58.000000000 -0500 Birth: 2026-01-05 00:07:59.000000000 -0500 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505017 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-05 00:08:03.000000000 -0500 Modify: 2026-01-05 00:08:03.000000000 -0500 Change: 2026-01-05 00:08:03.000000000 -0500 Birth: 2026-01-05 00:08:04.000000000 -0500 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505016 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-05 00:08:03.000000000 -0500 Modify: 2026-01-05 00:08:04.000000000 -0500 Change: 2026-01-05 00:08:04.000000000 -0500 Birth: 2026-01-05 00:08:04.000000000 -0500 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505020 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-05 00:08:09.000000000 -0500 Modify: 2026-01-05 00:08:10.000000000 -0500 Change: 2026-01-05 00:08:10.000000000 -0500 Birth: 2026-01-05 00:08:10.000000000 -0500 File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505019 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-05 00:08:09.000000000 -0500 Modify: 2026-01-05 00:08:10.000000000 -0500 Change: 2026-01-05 00:08:10.000000000 -0500 Birth: 2026-01-05 00:08:10.000000000 -0500 PASS 54 (26s) == sanityn test 55a: rename vs unlink target dir ========= 00:08:17 (1767589697) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (9s) == sanityn test 55b: rename vs unlink source dir ========= 00:08:26 (1767589706) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (8s) == sanityn test 55c: rename vs unlink orphan target dir == 00:08:34 (1767589714) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (14s) == sanityn test 55d: rename file vs link ================= 00:08:48 (1767589728) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (11s) == sanityn test 55e: rename race AB/BA under the same parent dir ========================================================== 00:08:59 (1767589739) SKIP: sanityn test_55e needs >= 2 MDTs SKIP 55e (1s) == sanityn test 55f: rename: (P1/A -> P2/B) race with (P2/B -> P1/A) ========================================================== 00:09:00 (1767589740) fail_loc=0x155 We survived after P1A_P2B/P2B_P1A race test fail_loc=0x0 PASS 55f (15s) == sanityn test 55g: rename: race with trylock =========== 00:09:15 (1767589755) mdt.lustre-MDT0000.enable_rename_trylock=1 fail_loc=0x155 mv: cannot move '/mnt/lustre2/d55g.sanityn/b1/b2' to a subdirectory of itself, '/mnt/lustre2/d55g.sanityn/a1/a2/a3/a4/b2' We survived race with trylock test fail_loc=0x0 PASS 55g (15s) == sanityn test 56a: test llverdev with single large stripe ========================================================== 00:09:30 (1767589770) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1767589771 chunksize: 4096 size: 16777216000000 write offset: 16384000000kB 1.38725 MB/s read offset: 16384000000kB inf MB/s llverdev: data verified successfully PASS 56a (48s) == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 00:10:18 (1767589818) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1767589820 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 3.2993 MB/s read offset: 32768000000kB inf MB/s llverdev: data verified successfully PASS 56b (43s) == sanityn test 60: Verify data_version behaviour ======== 00:11:01 (1767589861) PASS 60 (3s) == sanityn test 70a: cd directory && rm directory ======== 00:11:04 (1767589864) PASS 70a (3s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 00:11:07 (1767589867) PASS 70b (3s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 00:11:10 (1767589870) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0026762 s, 24.5 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 00:11:11 (1767589871) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00363243 s, 18.0 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 00:11:12 (1767589872) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 00:11:13 (1767589873) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00774319 s, 8.5 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (2s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 00:11:15 (1767589875) llite.lustre-ffff993903017800.xattr_cache=1 llite.lustre-ffff9939106e5800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (3s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 00:11:18 (1767589878) llite.lustre-ffff993903017800.xattr_cache=1 llite.lustre-ffff9939106e5800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (3s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 00:11:21 (1767589881) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00413116 s, 248 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00623422 s, 164 kB/s 272711: get lock2 272712: get lock1 272711: done 272711: get lock1 272712: done 272712: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 272712: cannot get lock2: Resource deadlock avoided 272712: exit rc=0 272711: done 272711: exit rc=0 PASS 74 (7s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 00:11:28 (1767589888) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0354112 s, 59.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179492 s, 58.4 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4193: 273292 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (8s) == sanityn test 76: Verify MDT open_files listing ======== 00:11:36 (1767589896) opening files: ........ closing files: ........ PASS 76 (28s) == sanityn test 77a: check FIFO NRS policy =============== 00:12:04 (1767589924) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.163088 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381974 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0476141 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0663538 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0634486 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0564637 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0703273 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0808708 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0877301 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0954609 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0930477 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0794446 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.093541 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.090195 s, 11.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff993903017800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0687234 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.083169 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0981863 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127347 s, 82.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162686 s, 64.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157684 s, 66.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156697 s, 66.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170196 s, 61.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022171 s, 47.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182892 s, 57.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209407 s, 50.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243781 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711701 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190498 s, 55.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105434 s, 99.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107016 s, 98.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764274 s, 137 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147454 s, 71.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff993903017800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736027 s, 142 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg454-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg454-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (4s) == sanityn test 77b: check CRR-N NRS policy ============== 00:12:08 (1767589928) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.189485 s, 88.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270401 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.042356 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0468469 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292686 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356889 s, 29.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0884438 s, 11.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104103 s, 10.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104088 s, 10.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101182 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0992631 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111417 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115404 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127159 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00416976 s, 251 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169747 s, 61.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0706541 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00386083 s, 272 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155433 s, 67.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00205766 s, 510 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14024 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15086 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.053558 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00414171 s, 253 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0508391 s, 20.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294711 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00287149 s, 365 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685791 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155147 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692252 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0471797 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188748 s, 55.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242648 s, 43.2 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.177459 s, 94.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0398445 s, 26.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159863 s, 65.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0354515 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317697 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0683302 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0488391 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0636435 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210093 s, 49.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179135 s, 58.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0805666 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279674 s, 37.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160921 s, 65.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244903 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01171 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0948565 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121816 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129682 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137663 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138869 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123849 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814036 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126694 s, 8.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff993903017800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff993903017800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137803 s, 7.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00930156 s, 113 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125468 s, 83.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00431091 s, 243 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00479015 s, 219 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021767 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815492 s, 129 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269641 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137975 s, 76.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743228 s, 141 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg454-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg454-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (6s) == sanityn test 77c: check ORR NRS policy ================ 00:12:14 (1767589934) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.221973 s, 75.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228909 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0411495 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.040305 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0415686 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0680746 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0548873 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0606303 s, 17.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff993903017800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff993903017800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.082584 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0728313 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0834489 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0952004 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0899292 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0777644 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186113 s, 56.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178828 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143169 s, 73.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707832 s, 148 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133298 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215784 s, 48.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122598 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033332 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272197 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774722 s, 135 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016709 s, 62.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204407 s, 51.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169364 s, 61.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870735 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145137 s, 7.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747544 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557655 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117566 s, 89.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00494378 s, 212 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff993903017800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff993903017800.lock_unused_count=2 removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg454-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg454-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.158656 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292543 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0474261 s, 22.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0553255 s, 19.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0646769 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0430223 s, 24.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0636637 s, 16.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0617289 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0584247 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.050866 s, 20.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff993903017800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108543 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11562 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108866 s, 9.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff993903017800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116419 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121227 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130851 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136186 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206737 s, 50.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0433466 s, 24.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304134 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341762 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189191 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0505389 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0485571 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272771 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0462233 s, 22.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227303 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189817 s, 55.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195223 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0472671 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235387 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241612 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0350144 s, 29.9 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.205374 s, 81.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155974 s, 67.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0550529 s, 19.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304062 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0619151 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.059179 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273662 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0654747 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0634547 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0366294 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.053896 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0843424 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100921 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10624 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101863 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0954092 s, 11.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff993903017800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104522 s, 10.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192776 s, 54.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148463 s, 70.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170118 s, 61.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197636 s, 53.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079575 s, 132 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158527 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865146 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219863 s, 47.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206289 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140172 s, 74.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00314912 s, 333 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203561 s, 51.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206829 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240227 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015702 s, 66.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841633 s, 125 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg454-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg454-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (6s) == sanityn test 77d: check TRR nrs policy ================ 00:12:20 (1767589940) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.133055 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0349614 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314965 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.048891 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247045 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0403632 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0668044 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0708669 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0481607 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0560813 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0805167 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106156 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1115 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133218 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173136 s, 60.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219579 s, 47.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235691 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.032603 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124593 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324968 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139432 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0395119 s, 26.5 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff993903017800.lock_unused_count=11 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0382817 s, 27.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00402118 s, 261 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0353564 s, 29.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108697 s, 96.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00550243 s, 191 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019655 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027743 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0052806 s, 199 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140043 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145744 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00338509 s, 310 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.209418 s, 80.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0353596 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0358188 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0633681 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0726208 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0760194 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0608121 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0774527 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0856875 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0930581 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0910997 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0909643 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0773947 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.087834 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0830298 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0781036 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.091574 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206488 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237248 s, 44.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132601 s, 79.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231466 s, 45.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194536 s, 53.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0038471 s, 273 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255893 s, 41.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149924 s, 69.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241068 s, 43.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014401 s, 72.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016162 s, 64.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020427 s, 51.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206012 s, 50.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810405 s, 129 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153928 s, 68.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131116 s, 80.0 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff993903017800.lock_unused_count=2 removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg454-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg454-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.18959 s, 88.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169759 s, 61.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0486291 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0518597 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0543752 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.049167 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0544348 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0726297 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.076353 s, 13.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff993903017800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff993903017800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0999418 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0965192 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102075 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101292 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0956891 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0986516 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10327 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114488 s, 9.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158489 s, 66.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137666 s, 76.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017032 s, 61.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169923 s, 61.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130915 s, 80.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105793 s, 99.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200724 s, 52.2 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff993903017800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff993903017800.lock_unused_count=5 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280741 s, 37.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139325 s, 75.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130902 s, 80.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00378882 s, 277 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213007 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283169 s, 37.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00919721 s, 114 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154342 s, 67.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00449505 s, 233 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg454-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg454-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (7s) == sanityn test 77e: check TBF NID nrs policy ============ 00:12:27 (1767589947) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.204.54@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.154425 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272106 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316283 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028492 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0705896 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0719545 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101728 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113641 s, 9.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119053 s, 8.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff993903017800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff993903017800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0908034 s, 11.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0871765 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114214 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119677 s, 8.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0840514 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0995995 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0912161 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131309 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.19111 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.193757 s, 5.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205204 s, 5.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200152 s, 5.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205115 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205171 s, 5.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207232 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208518 s, 5.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207237 s, 5.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218738 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.210599 s, 5.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208498 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215943 s, 4.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.214827 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211596 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.215035 s, 4.9 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.136089 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262115 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0782021 s, 13.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff993903017800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108053 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10641 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115421 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113607 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111418 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128422 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0788268 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0891066 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0909376 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0878746 s, 11.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff993903017800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125434 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169653 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119346 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146065 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163256 s, 64.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210764 s, 49.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322492 s, 32.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033738 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659964 s, 159 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0362682 s, 28.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365564 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029743 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301416 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310028 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0403888 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0475959 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336742 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0341952 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0397373 s, 26.4 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.177964 s, 94.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316139 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0784886 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0875745 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0793301 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0761553 s, 13.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff993903017800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.092078 s, 11.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0873437 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106962 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107294 s, 9.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0854284 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0929176 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100461 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101922 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0883907 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.093579 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102738 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153197 s, 68.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224615 s, 46.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184939 s, 56.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175202 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217084 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258623 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0050098 s, 209 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188258 s, 55.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff993903017800.lock_unused_count=2 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208331 s, 50.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185967 s, 56.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308938 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025177 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161166 s, 65.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179019 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253913 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00535948 s, 196 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.209073 s, 80.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0361682 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.044225 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0584428 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0507281 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0650358 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0706789 s, 14.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0776812 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0847538 s, 12.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0725274 s, 14.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff993903017800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0749426 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0952861 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107642 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124325 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109231 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135775 s, 7.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff993903017800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13021 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013162 s, 79.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174458 s, 60.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169863 s, 61.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194839 s, 53.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214517 s, 48.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021788 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237307 s, 44.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00500309 s, 210 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238641 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00393739 s, 266 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221555 s, 47.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00527751 s, 199 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010266 s, 102 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01533 s, 68.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121728 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225769 s, 46.4 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (10s) == sanityn test 77f: check TBF JobID nrs policy ========== 00:12:37 (1767589957) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.173854 s, 96.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270784 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0552723 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330942 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0605322 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.055616 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0557503 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0634174 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0564065 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0789077 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823829 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.091031 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0987009 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112803 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105252 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107511 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0977495 s, 10.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219358 s, 47.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198528 s, 52.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213665 s, 49.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018818 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236419 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766067 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231609 s, 45.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267414 s, 39.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191759 s, 54.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00549707 s, 191 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711724 s, 147 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262838 s, 39.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881596 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252235 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816359 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00473147 s, 222 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg454-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg454-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.151981 s, 110 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291549 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261022 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.05862 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0571671 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0665234 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0647991 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0878229 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0769976 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0772146 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0977881 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0986157 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0944773 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0844617 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.093082 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0868639 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100936 s, 10.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192387 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170009 s, 61.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145851 s, 71.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167388 s, 62.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173724 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157231 s, 66.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018208 s, 57.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160804 s, 65.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175267 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015925 s, 65.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234329 s, 44.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719378 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00434735 s, 241 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00362829 s, 289 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00542297 s, 193 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00465902 s, 225 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.154917 s, 108 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335302 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197821 s, 53.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.06994 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0808716 s, 13.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105471 s, 9.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109924 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104096 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0952594 s, 11.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111735 s, 9.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107468 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114018 s, 9.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff993903017800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff993903017800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110236 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120974 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124742 s, 8.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137023 s, 7.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137152 s, 7.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867447 s, 121 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221248 s, 47.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158796 s, 66.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224138 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184081 s, 57.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151997 s, 69.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167669 s, 62.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200267 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323906 s, 32.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266322 s, 39.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217887 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021559 s, 48.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257911 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214398 s, 48.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294427 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00503834 s, 208 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg454-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg454-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.17269 s, 97.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270734 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0601352 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0578574 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0734309 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0736965 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0625311 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0948539 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0956571 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103638 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0947587 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0811308 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0822039 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100605 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.085814 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0645703 s, 16.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff993903017800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff993903017800.lock_unused_count=1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1097 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114869 s, 91.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184143 s, 56.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013608 s, 77.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155153 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193253 s, 54.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153925 s, 68.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202112 s, 51.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017486 s, 60.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160459 s, 65.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175476 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00336399 s, 312 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00371347 s, 282 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111968 s, 93.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00932475 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00598932 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00146141 s, 718 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (9s) == sanityn test 77g: Change TBF type directly ============ 00:12:46 (1767589966) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (5s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 00:12:51 (1767589971) oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg454-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg454-client: oleg454-server: ssh exited with exit code 2 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Invalid argument oleg454-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Invalid argument oleg454-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Invalid argument oleg454-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 PASS 77h (5s) == sanityn test 77i: Change rank of TBF rule ============= 00:12:56 (1767589976) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (11s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 00:13:07 (1767589987) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.204861 s, 81.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026736 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321432 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296239 s, 35.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0775333 s, 13.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0770369 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.070236 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0713089 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0886082 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106648 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0977757 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0861371 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113015 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0987693 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102811 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0921692 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0891733 s, 11.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177809 s, 59.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262394 s, 40.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168622 s, 62.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214634 s, 48.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230407 s, 45.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274588 s, 38.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195716 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241502 s, 43.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00634388 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015026 s, 69.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792273 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00589356 s, 178 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00518192 s, 202 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00350673 s, 299 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00547613 s, 191 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00359365 s, 292 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.01821 s, 20.9 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4545 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (55s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 00:14:02 (1767590042) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg454-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg454-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg454-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg454-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.23978 s, 13.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0949715 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0724974 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0894345 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.067623 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0798327 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0976091 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0859997 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0970674 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131077 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131104 s, 8.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138497 s, 7.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136591 s, 7.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123314 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136029 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170904 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148471 s, 7.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240077 s, 43.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136933 s, 76.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251983 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228798 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125517 s, 83.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190956 s, 54.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292429 s, 35.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283106 s, 37.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251885 s, 41.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00404721 s, 259 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242021 s, 43.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00426492 s, 246 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256226 s, 40.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027407 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00490301 s, 214 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00356647 s, 294 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.7285 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9236 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg454-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg454-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg454-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg454-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.2392 s, 13.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0730853 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0762776 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0636476 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0910564 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0897705 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0815708 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.072349 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0818333 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128246 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128196 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115677 s, 9.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148774 s, 7.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115119 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12686 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146782 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146352 s, 7.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173176 s, 60.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00471428 s, 222 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245798 s, 42.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026007 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272075 s, 38.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172304 s, 60.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259115 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219187 s, 47.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225674 s, 46.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00378087 s, 277 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594333 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260523 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108931 s, 96.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259159 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00170327 s, 616 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00191905 s, 546 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.7337 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9154 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (138s) == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 00:16:20 (1767590180) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg454-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg454-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg454-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg454-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.23977 s, 13.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0580657 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0720026 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860489 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0800472 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0992768 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0890941 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0840663 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0815113 s, 12.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131139 s, 8.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131185 s, 8.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129301 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147268 s, 7.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137664 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139063 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220867 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.196537 s, 5.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222406 s, 47.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183162 s, 57.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242773 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218548 s, 48.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136169 s, 77.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291876 s, 35.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128191 s, 81.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169865 s, 61.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223983 s, 46.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231145 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00305689 s, 343 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217998 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0037058 s, 283 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00420193 s, 250 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260749 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274999 s, 38.1 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.7044 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9243 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg454-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg454-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg454-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg454-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 1.23941 s, 13.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0899035 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0903809 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0797034 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0918942 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0874848 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0818877 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0833153 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0883901 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132035 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132033 s, 7.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110685 s, 9.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119471 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148266 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138428 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140645 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12802 s, 8.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134151 s, 78.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254531 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280217 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305904 s, 34.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165858 s, 63.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018279 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304545 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141707 s, 74.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.032508 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264986 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290536 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068615 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276835 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281349 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00210381 s, 498 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00225771 s, 464 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.7235 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.922 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (139s) == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 00:18:39 (1767590319) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.308394 s, 54.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0358034 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0472577 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0697201 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0743986 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0720305 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0705734 s, 14.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114957 s, 9.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0987641 s, 10.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110652 s, 9.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0989605 s, 10.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0970567 s, 10.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0920798 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110731 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121581 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100989 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0989385 s, 10.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147709 s, 71.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02124 s, 49.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179626 s, 58.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021497 s, 48.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020159 s, 52.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203423 s, 51.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02182 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128357 s, 81.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190064 s, 55.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00919355 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00350729 s, 299 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268418 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581597 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105379 s, 99.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00413303 s, 254 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027981 s, 37.5 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg454-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg454-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.978 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75404 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.54@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.54@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.306455 s, 54.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0526042 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0484353 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0899406 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0900868 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0854651 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0854501 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0910224 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0744616 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0922587 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0887189 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0985161 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0941588 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213353 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223692 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260466 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.27733 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155052 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218639 s, 48.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319997 s, 32.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246306 s, 42.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280098 s, 37.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231169 s, 45.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00974321 s, 108 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188874 s, 55.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230137 s, 45.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158008 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118569 s, 88.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117391 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743251 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804999 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606358 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0044094 s, 238 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg454-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg454-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.05217 s, 20.8 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75083 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.54@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.170415 s, 98.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0399702 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0579201 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0456417 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.066327 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0577293 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0557217 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0566688 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0893447 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103534 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126488 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134336 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129535 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144355 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111258 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14973 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147717 s, 7.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026807 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230318 s, 45.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246274 s, 42.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272817 s, 38.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326346 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130522 s, 80.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241753 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0348336 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0360363 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309734 s, 33.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324114 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0383224 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103724 s, 101 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0386003 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01982 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142512 s, 73.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.9807 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.90085 s, 21.4 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.189818 s, 88.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0544803 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0585345 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0595077 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0409852 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282891 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0533325 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0613343 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0572789 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0552834 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0698405 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0765704 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0927418 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0849707 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103358 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0940642 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0890395 s, 11.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184405 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194557 s, 53.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220978 s, 47.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222476 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0434271 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0434382 s, 24.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257606 s, 40.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267111 s, 39.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294141 s, 35.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310535 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282058 s, 37.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308896 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0505405 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293213 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286604 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304432 s, 34.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.79571 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75274 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.96537 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75235 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.198947 s, 84.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.03617 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0325341 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101213 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0937227 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0853818 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0881079 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0800314 s, 13.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.098178 s, 10.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114299 s, 9.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109104 s, 9.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104341 s, 10.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.097785 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109125 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106115 s, 9.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100228 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100867 s, 10.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137187 s, 76.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222877 s, 47.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131872 s, 79.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0429934 s, 24.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191654 s, 54.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118587 s, 88.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268783 s, 39.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323295 s, 32.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.049543 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100249 s, 105 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0481682 s, 21.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0526546 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266875 s, 39.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270767 s, 38.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763129 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827728 s, 127 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg454-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg454-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.5205 s, 5.4 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4552 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.301428 s, 55.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0511917 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0770374 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0806306 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0706067 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104132 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0898028 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.090688 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0878138 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0983739 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0847634 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860098 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0884362 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0901481 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0970152 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0986811 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0904703 s, 11.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205343 s, 51.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179522 s, 58.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264177 s, 39.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192385 s, 54.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245835 s, 42.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027695 s, 37.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.010657 s, 98.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0334093 s, 31.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181727 s, 57.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206008 s, 50.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228979 s, 45.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252858 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233754 s, 44.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302461 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00586228 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0333159 s, 31.5 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg454-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg454-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.95996 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75314 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.18361 s, 91.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0391546 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030096 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0682521 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0773322 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100537 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0833249 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0762317 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0816374 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118213 s, 8.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109908 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0990214 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0948108 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10585 s, 9.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.097599 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0839968 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104465 s, 10.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209344 s, 50.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216525 s, 48.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233583 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243285 s, 43.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254773 s, 41.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260242 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0449323 s, 23.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284904 s, 36.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252445 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290577 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.051174 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317617 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309754 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316857 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0550883 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0389553 s, 26.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.85399 s, 10.6 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75343 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.80997 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75075 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (359s) == sanityn test 77kb: Check different granular TBF type combination: jobid+opcode ========================================================== 00:24:38 (1767590678) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.332861 s, 50.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0522745 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0575725 s, 18.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0698982 s, 15.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0772882 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0884558 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0958642 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0840707 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0809718 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0720166 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113647 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.089139 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112237 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0932278 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0918011 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109629 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104104 s, 10.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141461 s, 74.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172112 s, 60.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182259 s, 57.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205638 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288276 s, 36.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230722 s, 45.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063693 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029129 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230245 s, 45.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153319 s, 68.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025747 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193055 s, 54.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840159 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119606 s, 87.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00346049 s, 303 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00200603 s, 523 MB/s removed '/mnt/lustre/d77kb.sanityn/nrs_r_oleg454-client.virtnet' removed '/mnt/lustre/d77kb.sanityn/nrs_w_oleg454-client.virtnet' removed directory '/mnt/lustre/d77kb.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.03089 s, 20.8 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.7521 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kb (37s) == sanityn test 77kc: Check different granular TBF type combination: nid+opcode ========================================================== 00:25:15 (1767590715) ost.OSS.ost_io.nrs_policies=tbf nid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.204.54@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.204.54@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.204652 s, 82.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267819 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0481262 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0634123 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0836223 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0715992 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0570065 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0832213 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0922448 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0712951 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0540285 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0826843 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0924097 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0791662 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0848529 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100427 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0958788 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193599 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229769 s, 45.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022108 s, 47.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212723 s, 49.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163048 s, 64.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146978 s, 71.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0088033 s, 119 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124969 s, 83.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326116 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277922 s, 37.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217457 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116738 s, 89.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272088 s, 38.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684851 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606429 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944064 s, 111 MB/s removed '/mnt/lustre/d77kc.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77kc.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77kc.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.02482 s, 20.9 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75313 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kc.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kc (40s) == sanityn test 77kd: Check different granular TBF type combination: nid+jobid ========================================================== 00:25:55 (1767590755) ost.OSS.ost_io.nrs_policies=tbf nid+jobid ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.204.54@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kd.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.329685 s, 50.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0364925 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271079 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0873479 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0900863 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0899974 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117665 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10765 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104465 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107056 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112586 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119918 s, 8.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131725 s, 8.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104907 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116042 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12614 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114653 s, 9.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018897 s, 55.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239302 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224027 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159897 s, 65.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183818 s, 57.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265631 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189774 s, 55.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131724 s, 79.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164534 s, 63.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100391 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00537029 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239299 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00390424 s, 269 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104319 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281737 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123042 s, 85.2 MB/s removed '/mnt/lustre/d77kd.sanityn/nrs_r_oleg454-client.virtnet' removed '/mnt/lustre/d77kd.sanityn/nrs_w_oleg454-client.virtnet' removed directory '/mnt/lustre/d77kd.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kd.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.97606 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kd.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.94197 s, 21.2 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77kd.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kd (32s) == sanityn test 77ke: Check different granular TBF type combination: jobid+opcode ========================================================== 00:26:27 (1767590787) ost.OSS.ost_io.nrs_policies=tbf jobid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ke.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.21133 s, 79.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0388886 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321194 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0746361 s, 14.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0929222 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0986393 s, 10.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0967831 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10058 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0927097 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122897 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113333 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11198 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0980875 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116636 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126906 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0989176 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106264 s, 9.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189816 s, 55.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198906 s, 52.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201029 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219694 s, 47.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0058093 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300343 s, 34.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269413 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00390956 s, 268 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178415 s, 58.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204991 s, 51.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00538418 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968343 s, 108 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211397 s, 49.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142213 s, 73.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277504 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00606612 s, 173 MB/s removed '/mnt/lustre/d77ke.sanityn/nrs_r_oleg454-client.virtnet' removed '/mnt/lustre/d77ke.sanityn/nrs_w_oleg454-client.virtnet' removed directory '/mnt/lustre/d77ke.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ke.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.84326 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ke.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75181 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77ke.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.95361 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75137 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77ke.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77ke (77s) == sanityn test 77kf: Check different granular TBF type combination: uid+gid ========================================================== 00:27:44 (1767590864) ost.OSS.ost_io.nrs_policies=tbf uid+gid ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kf.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.206846 s, 81.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335151 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310465 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0627779 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0698328 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.092225 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0740259 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0785897 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0606568 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0764664 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0911431 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0842194 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0829588 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0938561 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0821094 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105212 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106294 s, 9.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193703 s, 54.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248626 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205271 s, 51.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186605 s, 56.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179038 s, 58.6 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00373535 s, 281 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215544 s, 48.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064169 s, 163 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249468 s, 42.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00417391 s, 251 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145066 s, 72.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238925 s, 43.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060769 s, 173 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126599 s, 82.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136118 s, 77.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0033652 s, 312 MB/s removed '/mnt/lustre/d77kf.sanityn/nrs_r_oleg454-client.virtnet' removed '/mnt/lustre/d77kf.sanityn/nrs_w_oleg454-client.virtnet' removed directory '/mnt/lustre/d77kf.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kf.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.5866 s, 5.4 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kf.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.923 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77kf.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kf (64s) == sanityn test 77kg: check different granular TBF type combination: uid+opcode ========================================================== 00:28:48 (1767590928) ost.OSS.ost_io.nrs_policies=tbf uid+opcode ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.302391 s, 55.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0407943 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0337803 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0879757 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0654475 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0848082 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0799481 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0773946 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0822104 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117249 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.097851 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111132 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106995 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0902485 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10269 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105885 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125573 s, 8.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158716 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165938 s, 63.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227281 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259181 s, 40.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217752 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221574 s, 47.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210555 s, 49.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176607 s, 59.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237468 s, 44.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226111 s, 46.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164728 s, 63.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103966 s, 101 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101176 s, 104 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143737 s, 73.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00514061 s, 204 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808179 s, 130 MB/s removed '/mnt/lustre/d77kg.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77kg.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77kg.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.97862 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75403 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/nrs_r_oleg454-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.205221 s, 81.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.043542 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0455514 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0582184 s, 18.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0807558 s, 13.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0674823 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100597 s, 10.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0802833 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0830594 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0748433 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0788536 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107933 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0985843 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.089114 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0824536 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103692 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102762 s, 10.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215999 s, 48.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0465522 s, 22.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232542 s, 45.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025833 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261096 s, 40.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269874 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272974 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292946 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312901 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322439 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314954 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.059783 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0361418 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0354699 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0359034 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0392053 s, 26.7 MB/s removed '/mnt/lustre/d77kg.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77kg.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77kg.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.84485 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75126 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77kg.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.80972 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77kg.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.75237 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77kg.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete PASS 77kg (113s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 00:30:41 (1767591041) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (5s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 00:30:46 (1767591046) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg454-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.33309 s, 315 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg454-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.0159 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (51s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 00:31:37 (1767591097) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.213811 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0387863 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033819 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329728 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0888798 s, 11.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122619 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1108 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114535 s, 9.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121059 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109096 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104832 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110915 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121012 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140402 s, 7.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff993903017800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff993903017800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120553 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130587 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141511 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013077 s, 80.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00978827 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114358 s, 91.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0114395 s, 91.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217665 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239176 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167329 s, 62.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227645 s, 46.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195893 s, 53.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200893 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201686 s, 52.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208733 s, 50.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222714 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00446653 s, 235 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00349516 s, 300 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787319 s, 133 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg454-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg454-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.96742 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.94448 s, 21.2 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.190161 s, 88.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.046822 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304647 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140194 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129986 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150992 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179701 s, 5.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147147 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149285 s, 7.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182789 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167726 s, 6.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185926 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.175071 s, 6.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.18552 s, 5.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160095 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169814 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.165081 s, 6.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220512 s, 47.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254304 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250535 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0436396 s, 24.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009975 s, 105 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310925 s, 33.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.041865 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0397038 s, 26.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0366942 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0360593 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139527 s, 75.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264584 s, 39.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026717 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00581324 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00614168 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121939 s, 86.0 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg454-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg454-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.03072 s, 20.8 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.94293 s, 21.2 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (72s) == sanityn test 77o: Changing rank should not panic ====== 00:32:49 (1767591169) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (5s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 00:32:54 (1767591174) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (52s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 00:33:46 (1767591226) mds.MDS.mdt.nrs_policies=tbf oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg454-client: oleg454-server: ssh exited with exit code 36 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg454-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg454-client: oleg454-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (17s) == sanityn test 77r: Change type of tbf policy at run time ========================================================== 00:34:03 (1767591243) mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 308343 kill md_thread xargs PASS 77r (42s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 00:34:45 (1767591285) oleg454-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device oleg454-server: error: set_param: setting 'ost/OSS/*/nrs_orr_quantum'='1': No such device pdsh@oleg454-client: oleg454-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (4s) == sanityn test 79: xattr: intent error ================== 00:34:49 (1767591289) File: /mnt/lustre/d79.sanityn Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272506522 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-05 00:34:50.000000000 -0500 Modify: 2026-01-05 00:34:50.000000000 -0500 Change: 2026-01-05 00:34:50.000000000 -0500 Birth: 2026-01-05 00:34:51.000000000 -0500 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (14s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 00:35:03 (1767591303) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (2s) == sanityn test 80b: Accessing directory during migration ========================================================== 00:35:05 (1767591305) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (2s) == sanityn test 81a: rename and stat under striped directory ========================================================== 00:35:07 (1767591307) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 00:35:08 (1767591308) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (2s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 00:35:10 (1767591310) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (2s) == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 00:35:12 (1767591312) - open/close 2043 (time 1767591324.95 total 10.00 last 204.23) total: 2500 open/close in 12.16 seconds: 205.53 ops/second snapshot_time 1767591353.392227444 secs.nsecs start_time 1767591314.747233788 secs.nsecs elapsed_time 38.644993656 secs.nsecs open 2505 samples [usecs] 108 10533 1529766 1484569976 close 2504 samples [usecs] 71 8830 493064 211893688 mknod 2501 samples [usecs] 189 10442 1340817 1231488847 unlink 2500 samples [usecs] 278 17965 1808097 2810187073 mkdir 2 samples [usecs] 1465 1768 3233 5272049 rmdir 2 samples [usecs] 3640 5256 8896 40875136 rename 2500 samples [usecs] 245 13677 3732173 9111883753 getattr 3761 samples [usecs] 9 1737 103136 8908408 parallel_rename_file 2500 samples [usecs] 245 13677 3732173 9111883753 crossdir_rename 2500 samples [usecs] 245 13677 3732173 9111883753 PASS 81d (42s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 00:35:54 (1767591354) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7553 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7553 PASS 82 (4s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 00:35:58 (1767591358) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 00:35:59 (1767591359) fail_loc=0x8000060b PASS 84 (9s) == sanityn test 85: Lustre API root cache race =========== 00:36:08 (1767591368) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1767591369 Finishing test test1 at 1767591370 PASS 85 (4s) == sanityn test 90: open/create and unlink striped directory ========================================================== 00:36:12 (1767591372) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 00:36:14 (1767591374) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 00:36:15 (1767591375) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (2s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 00:36:17 (1767591377) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5875 0x16f3 0x240000400 1 5848 0x16d8 0x280000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5876 0x16f4 0x240000400 1 5849 0x16d9 0x280000400 rc1=0 and rc2=0 PASS 93 (10s) == sanityn test 94: signal vs CP callback race =========== 00:36:28 (1767591388) osc.lustre-OST0000-osc-ffff993903017800.idle_timeout=0 osc.lustre-OST0000-osc-ffff9939106e5800.idle_timeout=0 osc.lustre-OST0001-osc-ffff993903017800.idle_timeout=0 osc.lustre-OST0001-osc-ffff9939106e5800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0627172 s, 131 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 2.99086 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5988: 326816 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0236936 s, 173 kB/s osc.lustre-OST0000-osc-ffff993903017800.idle_timeout=debug osc.lustre-OST0000-osc-ffff9939106e5800.idle_timeout=debug osc.lustre-OST0001-osc-ffff993903017800.idle_timeout=debug osc.lustre-OST0001-osc-ffff9939106e5800.idle_timeout=debug PASS 94 (10s) == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 00:36:38 (1767591398) llite.lustre-ffff993903017800.fast_read=0 llite.lustre-ffff9939106e5800.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0286535 s, 73.2 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff993903017800.fast_read=1 llite.lustre-ffff9939106e5800.fast_read=1 PASS 95a (16s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 00:36:54 (1767591414) llite.lustre-ffff993903017800.fast_read=0 llite.lustre-ffff9939106e5800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.00269911 s, 4.6 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff993903017800.fast_read=1 llite.lustre-ffff9939106e5800.fast_read=1 PASS 95b (9s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 00:37:03 (1767591423) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (2s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 00:37:05 (1767591425) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00217431 s, 1.9 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (4s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 00:37:09 (1767591429) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0904476 s, 23.2 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (3s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 00:37:12 (1767591432) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0908719 s, 23.1 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (4s) == sanityn test 100e: DoM: read on open and file size ==== 00:37:16 (1767591436) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.00316459 s, 3.8 MB/s PASS 100e (3s) == sanityn test 101a: Discard DoM data on unlink ========= 00:37:19 (1767591439) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000500282 s, 8.2 MB/s PASS 101a (3s) == sanityn test 101b: Discard DoM data on rename ========= 00:37:22 (1767591442) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000878968 s, 4.7 MB/s PASS 101b (3s) == sanityn test 101c: Discard DoM data on close-unlink === 00:37:25 (1767591445) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000499221 s, 8.2 MB/s PASS 101c (4s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 00:37:30 (1767591450) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1767591451 lockahead_test: lockahead_test.c:1171: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1767591454 Finishing test test23 at 1767591454 PASS 103 (8s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 00:37:38 (1767591458) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (2s) == sanityn test 105: Glimpse and lock cancel race ======== 00:37:40 (1767591460) fail_loc=0x416 PASS 105 (18s) == sanityn test 106a: Verify the btime via statx() ======= 00:37:58 (1767591478) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (2s) == sanityn test 106b: Glimpse RPCs test for statx ======== 00:38:00 (1767591480) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0033826 s, 303 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272509057 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-05 00:38:01.000000000 -0500 Modify: 2026-01-05 00:38:01.000000000 -0500 Change: 2026-01-05 00:38:01.000000000 -0500 Birth: 2026-01-05 00:38:02.000000000 -0500 /mnt/lustre/f106b.sanityn 144115205272509057 -rw-r--r-- 1 0 0 1767591482 1767591481 1767591481 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272509057 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-05 00:38:01.000000000 -0500 Modify: 2026-01-05 00:38:01.000000000 -0500 Change: 2026-01-05 00:38:01.000000000 -0500 Birth: 2026-01-05 00:38:02.000000000 -0500 1767591481 1024 11 PASS 106b (3s) == sanityn test 106c: Verify statx attributes mask ======= 00:38:03 (1767591483) PASS 106c (3s) == sanityn test 107a: Basic grouplock conflict =========== 00:38:06 (1767591486) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.10808 s, 97.0 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 5855 0x16df 0x280000400 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 First grouplock blocks second one PASS 107a (5s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 00:38:11 (1767591491) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.117626 s, 89.1 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5885 0x16fd 0x240000400 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7553 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (10s) == sanityn test 108a: lseek: parallel updates ============ 00:38:21 (1767591501) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.0024913 s, 4.0 MB/s llite.lustre-ffff993903017800.hybrid_io=0 llite.lustre-ffff9939106e5800.hybrid_io=0 fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 4.36859 s, 1.9 MB/s PASS 108a (7s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 00:38:28 (1767591508) 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg454-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg454-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg454-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg454-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg454-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg454-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg454-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg454-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg454-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg454-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg454-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg454-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg454-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg454-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg454-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 192.168.204.154@tcp:/lustre /mnt/lustre lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.204.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,encrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg454-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg454-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 114: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 156: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg454-server' oleg454-server: oleg454-server.virtnet: executing load_modules_local oleg454-server: Loading modules from /home/green/git/lustre-release/lustre oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1034: echo: write error: Device or resource busy oleg454-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 1035: echo: write error: Device or resource busy oleg454-server: detected 4 online CPUs by sysfs oleg454-server: MODOPTS_LIBCFS= oleg454-server: Force libcfs to create 2 CPU partitions Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre Starting client: oleg454-client.virtnet: -o user_xattr,flock 192.168.204.154@tcp:/lustre /mnt/lustre2 PASS 109 (587s) == sanityn test 110: do not grant another lock on resend ========================================================== 00:48:15 (1767592095) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-01-05 00:48:16.000000000 -0500 Modify: 2026-01-05 00:48:16.000000000 -0500 Change: 2026-01-05 00:48:16.000000000 -0500 Birth: 2026-01-05 00:48:17.000000000 -0500 fail_loc=0 fail_val=0 Connected clients: oleg454-client.virtnet oleg454-client.virtnet PASS 110 (81s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 00:49:36 (1767592176) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (2s) == sanityn test 112: update max-inherit in default LMV === 00:49:38 (1767592178) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 00:49:39 (1767592179) lustre-OST0000-osc-ffff993937d06000 active. lustre-OST0001-osc-ffff993937d06000 active. lustre-OST0000-osc-ffff993937d06000 active. lustre-OST0001-osc-ffff993937d06000 active. MGC192.168.204.154@tcp active. PASS 113 (3s) == sanityn test 114: implicit default LMV inherit ======== 00:49:42 (1767592182) SKIP: sanityn test_114 We need at least 2 MDTs for this test SKIP 114 (2s) == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 00:49:44 (1767592184) SKIP: sanityn test_115 ldiskfs only test SKIP 115 (1s) == sanityn test 116: DNE: Set default LMV layout from a remote client ========================================================== 00:49:45 (1767592185) SKIP: sanityn test_116 needs >= 2 MDTs SKIP 116 (2s) == sanityn test 117a: TCU: Init and enable Trash Can on MDTs ========================================================== 00:49:47 (1767592187) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_enable=0 PASS 117a (2s) == sanityn test 117b: Move regular file and empty dir into trash can dir ========================================================== 00:49:49 (1767592189) SKIP: sanityn test_117b ldiskfs only test SKIP 117b (1s) == sanityn test 117c: Move deleted tree with multiple levels into trash ========================================================== 00:49:50 (1767592190) SKIP: sanityn test_117c ldiskfs only test SKIP 117c (1s) == sanityn test 117d: Per-User Trash can Type testing ==== 00:49:51 (1767592191) SKIP: sanityn test_117d ldiskfs only test SKIP 117d (1s) == sanityn test 117e: Undeleted dir in trash should keep its original xattrs ========================================================== 00:49:52 (1767592192) SKIP: sanityn test_117e ldiskfs only test SKIP 117e (1s) == sanityn test 117f: Uncache the dentry under the trash dir ========================================================== 00:49:53 (1767592193) SKIP: sanityn test_117f ldiskfs only test SKIP 117f (1s) == sanityn test 117g: Access .Trash for a non-striped directory ========================================================== 00:49:54 (1767592194) SKIP: sanityn test_117g ldiskfs only test SKIP 117g (1s) == sanityn test 117h: Permission testing of various operations for files in Trash Can ========================================================== 00:49:55 (1767592195) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain Trash Can Type: plain Trash list: ===> /mnt/lustre/.lustre/trash/MDT0000: 0x200000403:0x5:0x0 /mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0: tdir1 tfile3 /mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0/tdir1: <=== touch: setting times of '/mnt/lustre/.lustre/trash/f117h.sanityn.crfail': No such file or directory mkdir: cannot create directory '/mnt/lustre/.lustre/trash/f117h.sanityn.crfail': Operation not permitted touch: setting times of '/mnt/lustre/.lustre/trash/MDT0000/f117h.sanityn.crfail': No such file or directory mkdir: cannot create directory '/mnt/lustre/.lustre/trash/MDT0000/f117h.sanityn.crfail': Operation not permitted rmdir: failed to remove '/mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0': Operation not permitted touch: setting times of '/mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0/f117h.sanityn.crfail': No such file or directory mv: cannot move '/mnt/lustre/d117h.sanityn/f117h.sanityn.mvfail' to '/mnt/lustre/.lustre/trash/f117h.sanityn.mvfail': Operation not permitted mv: cannot move '/mnt/lustre/d117h.sanityn/f117h.sanityn.mvfail' to '/mnt/lustre/.lustre/trash/MDT0000/f117h.sanityn.mvfail': No data available mv: cannot move '/mnt/lustre/d117h.sanityn/f117h.sanityn.mvfail' to '/mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0/f117h.sanityn.mvfail': No data available mv: cannot move '/mnt/lustre/d117h.sanityn/d117h.sanityn.mvfail' to '/mnt/lustre/.lustre/trash/d117h.sanityn.mvfail': Operation not permitted mv: cannot move '/mnt/lustre/d117h.sanityn/d117h.sanityn.mvfail' to '/mnt/lustre/.lustre/trash/MDT0000/d117h.sanityn.mvfail': No data available mv: cannot move '/mnt/lustre/d117h.sanityn/d117h.sanityn.mvfail' to '/mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0/d117h.sanityn.mvfail': No data available chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0//tfile3 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0//tfile3 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0//tfile3 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0//tfile3 chmod: changing permissions of '/mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0//tfile3': Bad file descriptor truncate: cannot truncate '/mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0//tfile3' to length 4096: Bad file descriptor chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0//tdir1 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0//tdir1 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0//tdir1 chattr: Bad file descriptor while setting flags on /mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0//tdir1 chmod: changing permissions of '/mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0//tdir1': Bad file descriptor setfattr: /mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0//tfile3: Bad file descriptor setfattr: /mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0//tfile3: Bad file descriptor setfattr: /mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0//tdir1: Bad file descriptor setfattr: /mnt/lustre/.lustre/trash/MDT0000/0x200000403:0x5:0x0//tdir1: Bad file descriptor open: Bad file descriptor open: Bad file descriptor read: Operation not permitted mmap: Operation not permitted open: Is a directory open: Is a directory mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid PASS 117h (7s) == sanityn test 117i: Forbid to create a file with name .Trash ========================================================== 00:50:02 (1767592202) mkdir: cannot create directory '/mnt/lustre/d117i.sanityn/.Trash': Invalid argument touch: setting times of '/mnt/lustre/d117i.sanityn/.Trash': No such file or directory open(O_RDWR|O_CREAT): Invalid argument mcreate: cannot create `/mnt/lustre/d117i.sanityn/.Trash' with mode 0100644: Invalid argument ln: failed to create hard link '/mnt/lustre/d117i.sanityn/.Trash' => '/mnt/lustre/d117i.sanityn/f117i.sanityn': Invalid argument ln: failed to create symbolic link '/mnt/lustre/d117i.sanityn/.Trash': Invalid argument mv: cannot move '/mnt/lustre/d117i.sanityn/f117i.sanityn' to a subdirectory of itself, '/mnt/lustre/d117i.sanityn/.Trash' mv: cannot move '/mnt/lustre/d117i.sanityn/f117i.sanityn' to a subdirectory of itself, '/mnt/lustre/.Trash' ln: failed to create symbolic link '/mnt/lustre/d117i.sanityn/.Trash': Invalid argument mv: cannot move '/mnt/lustre/d117i.sanityn/d117i.sanityn' to a subdirectory of itself, '/mnt/lustre/d117i.sanityn/.Trash' mv: cannot move '/mnt/lustre/d117i.sanityn/d117i.sanityn' to a subdirectory of itself, '/mnt/lustre/.Trash' PASS 117i (3s) == sanityn test 117j: Move back file from Trash Can ====== 00:50:06 (1767592206) SKIP: sanityn test_117j ldiskfs only test SKIP 117j (2s) == sanityn test 117k: Unrm file from Trash Can =========== 00:50:07 (1767592207) SKIP: sanityn test_117k ldiskfs only test SKIP 117k (1s) == sanityn test 117l: Clean up files in Trash Can via command 'lfs trash clean' ========================================================== 00:50:08 (1767592208) SKIP: sanityn test_117l ldiskfs only test SKIP 117l (2s) == sanityn test 117m: Empty Trash Can via the command 'lfs trash clean' ========================================================== 00:50:10 (1767592210) SKIP: sanityn test_117m ldiskfs only test SKIP 117m (2s) == sanityn test 117n: Update LinkEA accordingly when moving file into Trash Can ========================================================== 00:50:12 (1767592212) SKIP: sanityn test_117n ldiskfs only test SKIP 117n (1s) == sanityn test 117o: Set/Clear NOTRASH flag on file ===== 00:50:13 (1767592213) SKIP: sanityn test_117o ldiskfs only test SKIP 117o (2s) == sanityn test 117p: Trash Can handling for remote dir/file ========================================================== 00:50:15 (1767592215) SKIP: sanityn test_117p needs >= 2 MDTs SKIP 117p (1s) == sanityn test 117pb: Trash Can handling for remote dir/file with multiple levels ========================================================== 00:50:16 (1767592216) SKIP: sanityn test_117pb needs >= 2 MDTs SKIP 117pb (2s) == sanityn test 117q: Access Trash Can via '.Trash' for striped directory ========================================================== 00:50:18 (1767592218) SKIP: sanityn test_117q needs >= 2 MDTs SKIP 117q (1s) == sanityn test 117r: Verify the deleting a striped directory on Trash Can ========================================================== 00:50:19 (1767592219) SKIP: sanityn test_117r ldiskfs only test SKIP 117r (2s) == sanityn test 117s: Create trash shard if it does not exist ========================================================== 00:50:21 (1767592221) SKIP: sanityn test_117s ldiskfs only test SKIP 117s (1s) == sanityn test 117t: UID/GID/ProjID for files in Trash Can ========================================================== 00:50:23 (1767592223) SKIP: sanityn test_117t ldiskfs only test SKIP 117t (1s) == sanityn test 117u: statfs() call should not account the space in Trash Can ========================================================== 00:50:24 (1767592224) SKIP: sanityn test_117u ldiskfs only test SKIP 117u (2s) == sanityn test 117ub: statfs() account the inode usage in Trash Can ========================================================== 00:50:26 (1767592226) SKIP: sanityn test_117ub ldiskfs only test SKIP 117ub (2s) == sanityn test 117v: Test for Trash Can UID/GID/PROJID setting ========================================================== 00:50:28 (1767592228) SKIP: sanityn test_117v ldiskfs only test SKIP 117v (3s) == sanityn test 117w: Test Trash Can with original UID/GID/PROJID ========================================================== 00:50:31 (1767592231) SKIP: sanityn test_117w ldiskfs only test SKIP 117w (1s) == sanityn test 117x: Unrm Files from Trash Can should also restore UID/GID/PROJID ========================================================== 00:50:32 (1767592232) SKIP: sanityn test_117x ldiskfs only test SKIP 117x (2s) == sanityn test 117ya: Test with symbol links with Trash Can ========================================================== 00:50:34 (1767592234) SKIP: sanityn test_117ya ldiskfs only test SKIP 117ya (1s) == sanityn test 117yb: Compare the directories containing symlinks after unrm recover ========================================================== 00:50:35 (1767592235) SKIP: sanityn test_117yb ldiskfs only test SKIP 117yb (2s) == sanityn test 117yc: Restoring UID/GID/PROJID for symlinks works ========================================================== 00:50:37 (1767592237) SKIP: sanityn test_117yc ldiskfs only test SKIP 117yc (1s) == sanityn test 117za: Test Trash Can for UID Trash Can type with nodemap ========================================================== 00:50:38 (1767592238) SKIP: sanityn test_117za ldiskfs only test SKIP 117za (2s) == sanityn test 117zb: Test Trash Can for PLAIN Trash Can type with nodemap ========================================================== 00:50:40 (1767592240) SKIP: sanityn test_117zb ldiskfs only test SKIP 117zb (2s) == sanityn test 117zc: TCU: Test striped dir for UID Trash Can type ========================================================== 00:50:42 (1767592242) SKIP: sanityn test_117zc ldiskfs only test SKIP 117zc (1s) == sanityn test 117zd: TCU: Test striped dir for UID Trash Can type with nodemap ========================================================== 00:50:43 (1767592243) On MGS 192.168.204.154, active = nodemap.active=1 waiting 10 secs for sync On MGS 192.168.204.154, tnm.trusted_nodemap = nodemap.tnm.trusted_nodemap=1 waiting 10 secs for sync nodemap.tnm.idmap= [ { idtype: uid, client_id: 500, fs_id: 10000 }, { idtype: uid, client_id: 501, fs_id: 10001 }, { idtype: uid, client_id: 502, fs_id: 10002 }, { idtype: uid, client_id: 503, fs_id: 10003 }, { idtype: uid, client_id: 504, fs_id: 10004 }, { idtype: uid, client_id: 505, fs_id: 10005 }, { idtype: uid, client_id: 506, fs_id: 10006 }, { idtype: uid, client_id: 507, fs_id: 10007 }, { idtype: uid, client_id: 508, fs_id: 10008 }, { idtype: uid, client_id: 509, fs_id: 10009 } ] SKIP: sanityn test_117zd ldiskfs only test On MGS 192.168.204.154, active = nodemap.active=0 waiting 10 secs for sync SKIP 117zd (39s) == sanityn test 118Aa: TCU: Orphan object handling for a regular file or an empty dir ========================================================== 00:51:22 (1767592282) SKIP: sanityn test_118Aa ldiskfs only test SKIP 118Aa (4s) == sanityn test 118Ab: TCU: Orphan object handling for non-emtpy dir ========================================================== 00:51:26 (1767592286) SKIP: sanityn test_118Ab ldiskfs only test SKIP 118Ab (2s) == sanityn test 118Ba: TCU: 'oldest' repeated deletion for a regular file or empty dir ========================================================== 00:51:28 (1767592288) SKIP: sanityn test_118Ba ldiskfs only test SKIP 118Ba (2s) == sanityn test 118Bb: TCU: 'timestamp' repeated deletion policy ========================================================== 00:51:30 (1767592290) SKIP: sanityn test_118Bb ldiskfs only test SKIP 118Bb (2s) == sanityn test 118C: TCU: add support for rename() ====== 00:51:32 (1767592292) SKIP: sanityn test_118C ldiskfs only test SKIP 118C (1s) == sanityn test 118d: TCU: repeated deletion of same fname for a empty dir in DNE env ========================================================== 00:51:33 (1767592293) SKIP: sanityn test_118d ldiskfs only test SKIP 118d (2s) == sanityn test 118e: TCU: Repeated deletion of same name for a non-empty remote dir ========================================================== 00:51:35 (1767592295) SKIP: sanityn test_118e ldiskfs only test SKIP 118e (1s) == sanityn test 118f: Avoid caching negative dentry cache for dir in Trash Can ========================================================== 00:51:36 (1767592296) SKIP: sanityn test_118f ldiskfs only test SKIP 118f (1s) == sanityn test 118g: TCU: Tolerate ENOSPC error bypass Trash Can ========================================================== 00:51:37 (1767592297) mdd.lustre-MDT0000.trash_can_enable=1 mdd.lustre-MDT0000.trash_can_type=plain Trash Can Type: plain UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 561959 14471 547488 3% /mnt/lustre[MDT:0] lustre-OST0000_UUID 119211 2443 116768 3% /mnt/lustre[OST:0] lustre-OST0001_UUID 118567 2439 116128 3% /mnt/lustre[OST:1] filesystem_summary: 247367 14471 232896 6% /mnt/lustre - create 10000 (time 1767592310.98 total 9.99 last 1001.45) - create 19995 (time 1767592320.98 total 19.99 last 999.40) - create 28841 (time 1767592330.98 total 29.99 last 884.57) - create 37931 (time 1767592340.98 total 39.99 last 908.98) - create 47918 (time 1767592350.98 total 49.99 last 998.66) - create 58091 (time 1767592360.98 total 59.99 last 1017.28) - create 67063 (time 1767592370.98 total 69.99 last 897.11) - create 70000 (time 1767592374.08 total 73.09 last 946.85) - create 78564 (time 1767592384.08 total 83.09 last 856.37) - create 87345 (time 1767592394.08 total 93.09 last 878.03) - create 90000 (time 1767592397.33 total 96.34 last 817.70) - create 98698 (time 1767592407.33 total 106.34 last 869.80) - create 107324 (time 1767592417.33 total 116.34 last 862.60) - create 110000 (time 1767592420.29 total 119.30 last 904.54) total: 110737 create in 120.00 seconds: 922.80 ops/second - unlinked 0 (time 1767592422 ; total 0 ; last 0) - unlinked 10000 (time 1767592442 ; total 20 ; last 20) - unlinked 20000 (time 1767592463 ; total 41 ; last 21) - unlinked 30000 (time 1767592485 ; total 63 ; last 22) - unlinked 40000 (time 1767592507 ; total 85 ; last 22) - unlinked 50000 (time 1767592530 ; total 108 ; last 23) - unlinked 60000 (time 1767592553 ; total 131 ; last 23) - unlinked 70000 (time 1767592610 ; total 188 ; last 57) - unlinked 80000 (time 1767592632 ; total 210 ; last 22) - unlinked 90000 (time 1767592657 ; total 235 ; last 25) - unlinked 100000 (time 1767592680 ; total 258 ; last 23) - unlinked 110000 (time 1767592708 ; total 286 ; last 28) unlink(/mnt/lustre/d118g.sanityn/tf110737) error: No such file or directory total: 110737 unlinks in 288 seconds: 384.503479 unlinks/second lfs trash trash: failed to open by FID:[0x200000002:0x4:0x0]: Stale file handle (116) sanityn test_118g: @@@@@@ FAIL: failed to clean Trash /mnt/lustre/d118g.sanityn Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7318:error() = /home/green/git/lustre-release/lustre/tests/sanityn.sh:9525:test_118g() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7694:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7757:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7572:run_test() = /home/green/git/lustre-release/lustre/tests/sanityn.sh:9536:main() Dumping lctl log to /tmp/testlogs//sanityn.test_118g.*.1767592711.log rsync: chown "/tmp/testlogs/.sanityn.test_118g.debug_log.oleg454-server.1767592711.log.V79qau" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.sanityn.test_118g.dmesg.oleg454-server.1767592711.log.HW8aKl" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.3] unlink(/mnt/lustre/d118g.sanityn/tf0) error: No such file or directory total: 0 unlinks in 0 seconds: -nan unlinks/second mdd.lustre-MDT0000.trash_can_enable=0 mdd.lustre-MDT0000.trash_can_type=uid lfs trash trash: failed to open by FID:[0x200000002:0x4:0x0]: Stale file handle (116) FAIL 118g (418s) == sanityn test 118h: TCU: run mdtest with TCU enabled === 00:58:37 (1767592717) SKIP: sanityn test_118h mdtest not found SKIP 118h (1s) == sanityn test 118i: TCU: trusted.unrm xattr should not be cached on client-side ========================================================== 00:58:38 (1767592718) SKIP: sanityn test_118i ldiskfs only test SKIP 118i (1s) == sanityn test 118j: TCU: access .Trash from users with different UID ========================================================== 00:58:40 (1767592720) SKIP: sanityn test_118j ldiskfs only test SKIP 118j (2s) == sanityn test 118k: TCU: acess .Trash from users with different NODEMAP UIDs ========================================================== 00:58:41 (1767592721) SKIP: sanityn test_118k ldiskfs only test SKIP 118k (1s) == sanityn test 200: service remains healthy while able to process request ========================================================== 00:58:42 (1767592722) at_max=10 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=19 fail_loc=0x1e1 fail_val=20 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.0959 s, 0.2 kB/s4096 bytes (4.1 kB, 4.0 KiB) copied, 20.0928 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.1131 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.1045 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.1251 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.1079 s, 0.2 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.129 s, 0.2 kB/s 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.1125 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 20.1217 s, 0.2 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1344 s, 0.1 kB/s4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1517 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1631 s, 0.1 kB/s 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1418 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1728 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1533 s, 0.1 kB/s 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1552 s, 0.1 kB/s1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1703 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 40.1601 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.2093 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.2214 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.2267 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.2738 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.2327 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.2286 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.235 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.273 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 60.232 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 80.3 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 80.3662 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 80.3443 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 80.3587 s, 0.1 kB/s4096 bytes (4.1 kB, 4.0 KiB) copied, 80.3786 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 80.3808 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 80.3611 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 80.3558 s, 0.1 kB/s 4096 bytes (4.1 kB, 4.0 KiB) copied, 80.358 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 100.442 s, 0.0 kB/s4096 bytes (4.1 kB, 4.0 KiB) copied, 100.411 s, 0.0 kB/s at_max=600 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=116 oleg454-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff99390612d000.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff99390612d000.ost_server_uuid in IDLE state after 0 sec PASS 200 (107s) cleanup: ====================================================== == sanityn test complete, duration 6842 sec ============== 01:00:30 (1767592830) sanityn: FAIL: test_118g failed to clean Trash /mnt/lustre/d118g.sanityn === sanityn: start cleanup 01:00:31 (1767592831) === Stopping clients: oleg454-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg454-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 01:01:17 (1767592877) ===