-----============= acceptance-small: sanityn ============----- Tue Apr 14 14:05:15 EDT 2026 oleg314-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Started clients oleg314-client.virtnet: 192.168.203.114@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt) oleg314-client.virtnet: executing check_config_client /mnt/lustre oleg314-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg314-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff964048299800.idle_timeout=debug osc.lustre-OST0000-osc-ffff964060cdb800.idle_timeout=debug osc.lustre-OST0001-osc-ffff964048299800.idle_timeout=debug osc.lustre-OST0001-osc-ffff964060cdb800.idle_timeout=debug disable quota as required oleg314-server: oleg314-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0357652 s, 29.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f7142] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 14:06:01 (1776189961) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (6s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 14:06:07 (1776189967) -rw-r--r-- 1 root root 0 Apr 14 14:06 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (7s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 14:06:14 (1776189974) -rw-r--r-- 1 root root 0 Apr 14 14:06 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (5s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 14:06:19 (1776189979) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (6s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 14:06:25 (1776189985) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (5s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 14:06:31 (1776189991) total 1 -rwxrwxrwx 1 root root 0 Apr 14 14:06 f2a -rwxrwxrwx 1 root root 0 Apr 14 14:06 f2b total 1 -rwxrwxrwx 1 root root 0 Apr 14 14:06 f2a -rwxrwxrwx 1 root root 0 Apr 14 14:06 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.0368166 s, 13.9 kB/s PASS 2e (6s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 14:06:37 (1776189997) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (2s) == sanityn test 2g: check blocks update on sync write ==== 14:06:39 (1776189999) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.30892 s, 6.8 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (6s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 14:06:46 (1776190006) PASS 3 (6s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 14:06:52 (1776190012) PASS 4 (9s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 14:07:01 (1776190021) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (7s) == sanityn test 6: remove of open file on other node ============================================================================ 14:07:08 (1776190028) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (5s) == sanityn test 7: remove of open directory on other node ======================================================================= 14:07:13 (1776190033) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (5s) == sanityn test 8: remove of open special file on other node ==================================================================== 14:07:19 (1776190039) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (5s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 14:07:24 (1776190044) PASS 9a (7s) == sanityn test 9b: append to striped sparse file ======== 14:07:31 (1776190051) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.058854 s, 17.8 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.0743047 s, 0.0 kB/s Data read (expecting 'foo'): foo PASS 9b (6s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 14:07:37 (1776190057) 1+0 records in 1+0 records out 1 byte copied, 0.00423068 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00611429 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.016083 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00719846 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0206497 s, 0.0 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00422296 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00956836 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00961284 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00783852 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00544618 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00614662 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00690792 s, 0.1 kB/s PASS 10a (7s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 14:07:44 (1776190064) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.00849283 s, 362 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0356473 s, 115 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.000236589 s, 13.0 MB/s PASS 10b (5s) == sanityn test 11: execution of file opened for write should return error ============================================================== 14:07:49 (1776190069) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (5s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 14:07:54 (1776190074) warning: '-runtime' deprecated, use '-t runtime' instead - link 451 (time 1776190086.31 total 10.01 last 45.06) - link 853 (time 1776190096.32 total 20.02 last 40.16) - link 1274 (time 1776190106.34 total 30.03 last 42.04) - link 1651 (time 1776190116.35 total 40.04 last 37.67) - link 2012 (time 1776190126.38 total 50.07 last 35.99) - link 2367 (time 1776190136.40 total 60.10 last 35.42) - link 2751 (time 1776190146.43 total 70.13 last 38.28) - link 3116 (time 1776190156.45 total 80.15 last 36.43) - link 3535 (time 1776190166.47 total 90.16 last 41.82) total: 3881 link in 100.01 seconds: 38.80 ops/second using seed 119289473 running for 100 seconds - stat 10000 (time 1776190087 ; total 10 ; last 10) - stat 19558 (time 1776190098 ; total 21 ; last 11) - stat 20000 (time 1776190098 ; total 21 ; last 0) - stat 30000 (time 1776190108 ; total 31 ; last 10) - stat 39501 (time 1776190119 ; total 42 ; last 11) - stat 40000 (time 1776190119 ; total 42 ; last 0) - stat 48096 (time 1776190130 ; total 53 ; last 11) - stat 50000 (time 1776190134 ; total 57 ; last 4) - stat 54797 (time 1776190145 ; total 68 ; last 11) - stat 59619 (time 1776190156 ; total 79 ; last 11) - stat 60000 (time 1776190156 ; total 79 ; last 0) - stat 65090 (time 1776190167 ; total 90 ; last 11) total: 69643 stats in 100 seconds: 696.429993 stats/second - unlinked 0 (time 1776190127 ; total 0 ; last 0) unlink(/mnt/lustre2/lockdir/lockfile3881) error: No such file or directory total: 3880 unlinks in 88 seconds: 44.090908 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (19714) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (19717) - No such process PASS 12 (144s) == sanityn test 13: test directory page revocation ======= 14:10:18 (1776190218) f13 f13 PASS 13 (7s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 14:10:25 (1776190225) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4309: 20888 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (5s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 14:10:30 (1776190230) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4309: 21461 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (4s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 14:10:35 (1776190235) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 22034 Terminated $DIR1/$tdir/sleep 60 PASS 14b (5s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 14:10:39 (1776190239) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 22609 Terminated $DIR1/$tdir/sleep 60 PASS 14c (4s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 14:10:44 (1776190244) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 23185 Terminated $DIR1/$tdir/sleep 60 PASS 14d (6s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 14:10:50 (1776190250) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7527424 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7527424kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (18s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 14:11:08 (1776190268) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.347969 s, 6.0 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.221718 s, 9.5 MB/s Chance of close/open is 1 in 50 Seed set to 271 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read 000003[1] 1776190272.198695 write 0x44790e thru 0x44f20a (0x78fd bytes) 000006[1] 1776190272.259640 write 0x4de3f4 thru 0x4e5960 (0x756d bytes) 000009[0] 1776190272.327802 write 0x9bf64f thru 0x9c9dc1 (0xa773 bytes) 000010[1] 1776190272.402827 mapwrite 0x398bd4 thru 0x39a130 (0x155d bytes) 000012[0] 1776190272.660548 write 0x09d276 thru 0x0a0ae0 (0x386b bytes) 000013[1] 1776190272.713996 write 0x71428d thru 0x71a3a6 (0x611a bytes) truncating to largest ever: 0x2f26e0 000015[0] 1776190272.791803 trunc from 0x9c9dc2 to 0x2f26df (0x6d76e2 bytes) 000016[1] 1776190272.872716 read 0x29e003 thru 0x2a3d7d (0x5d7b bytes) truncating to largest ever: 0x654da1 000017[1] 1776190272.928706 trunc from 0x2f26e0 to 0x654da0 (0x3626c1 bytes) 000018[1] 1776190273.019440 write 0x4c9443 thru 0x4cbd7c (0x293a bytes) 000022[0] 1776190273.153682 mapread 0x311b4e thru 0x31aaaf (0x8f62 bytes) 000024[0] 1776190273.248598 read 0x39643b thru 0x3a5bf9 (0xf7bf bytes) truncating to largest ever: 0x92a59c 000026[1] 1776190273.299560 trunc from 0x654da1 to 0x92a59b (0x2d57fb bytes) 000027[1] 1776190273.359207 trunc from 0x92a59c to 0x76491a (0x1c5c81 bytes) 000028[0] 1776190273.408109 write 0x245192 thru 0x247e10 (0x2c7f bytes) 000029[1] 1776190273.447447 trunc from 0x76491b to 0x2e7155 (0x47d7c5 bytes) 000031[0] 1776190273.514847 trunc from 0x2e7156 to 0x290afa (0x5665b bytes) 000036[0] 1776190273.642537 write 0x9cb380 thru 0x9d4939 (0x95ba bytes) 000038[0] 1776190273.667679 trunc from 0x9d493a to 0x5ba9e3 (0x419f56 bytes) 000040[0] 1776190273.714472 mapwrite 0x723499 thru 0x72e17f (0xace7 bytes) 000041[0] 1776190273.877436 mapwrite 0x6b89a2 thru 0x6bda50 (0x50af bytes) 000042[0] 1776190274.037948 mapwrite 0x79616a thru 0x7a1d7f (0xbc16 bytes) 000046[1] 1776190274.265787 trunc from 0x7a1d80 to 0x32b862 (0x47651d bytes) 000047[1] 1776190274.391635 mapwrite 0x465216 thru 0x465d34 (0xb1f bytes) 000048[0] 1776190274.612882 write 0x0a002a thru 0x0ae535 (0xe50c bytes) 000049[0] 1776190274.673715 mapwrite 0x9a9874 thru 0x9abbb0 (0x233d bytes) 000050[1] 1776190274.928412 read 0x0d71ef thru 0x0e42c2 (0xd0d4 bytes) 000051[0] 1776190275.032997 write 0x80ffde thru 0x817b42 (0x7b65 bytes) 000052[0] 1776190275.091135 write 0x197786 thru 0x19cd54 (0x55cf bytes) 000054[0] 1776190275.120992 mapwrite 0x011be9 thru 0x016668 (0x4a80 bytes) 000055[1] 1776190275.448903 read 0x667c0a thru 0x66c458 (0x484f bytes) 000056[0] 1776190275.532741 mapread 0x4bb6fc thru 0x4c5682 (0x9f87 bytes) 000057[1] 1776190275.559500 mapwrite 0x3deb70 thru 0x3e48cd (0x5d5e bytes) 000059[1] 1776190275.832756 trunc from 0x9abbb1 to 0x8bdeda (0xedcd6 bytes) 000060[0] 1776190275.968781 read 0x5902cd thru 0x592270 (0x1fa4 bytes) 000061[1] 1776190276.066786 mapwrite 0x920160 thru 0x920fb2 (0xe53 bytes) 000064[1] 1776190276.234744 write 0x7fa3f7 thru 0x7fcc4b (0x2855 bytes) 000066[1] 1776190276.298790 mapwrite 0x5f4e1c thru 0x5fcadb (0x7cc0 bytes) 000067[1] 1776190276.627480 mapwrite 0x4b5fee thru 0x4bb7b8 (0x57cb bytes) 000068[0] 1776190276.872747 read 0x1f8506 thru 0x1fba0b (0x3506 bytes) 000069[0] 1776190276.909686 write 0x158645 thru 0x15c417 (0x3dd3 bytes) 000070[0] 1776190276.977195 mapread 0x266eca thru 0x2739f7 (0xcb2e bytes) 000074[0] 1776190277.313302 write 0x0293c0 thru 0x02fa44 (0x6685 bytes) 000076[1] 1776190277.366760 mapread 0x553a67 thru 0x555e7a (0x2414 bytes) 000077[0] 1776190277.419951 mapread 0x5fe45e thru 0x600a1d (0x25c0 bytes) 000078[0] 1776190277.498429 read 0x257617 thru 0x25ff4f (0x8939 bytes) 000079[0] 1776190277.637935 write 0x0466d4 thru 0x049359 (0x2c86 bytes) 000081[1] 1776190277.670491 trunc from 0x920fb3 to 0x8446e4 (0xdc8ce bytes) 000083[0] 1776190277.812344 write 0x531828 thru 0x53ff16 (0xe6ef bytes) 000085[0] 1776190277.905100 trunc from 0x8446e5 to 0x4e4852 (0x35fe92 bytes) 000086[0] 1776190277.954990 mapread 0x2c9dc1 thru 0x2cea9f (0x4cdf bytes) 000087[0] 1776190278.020547 mapread 0x0051e9 thru 0x00562f (0x447 bytes) 000088[0] 1776190278.029679 read 0x12968d thru 0x1341e6 (0xab5a bytes) 000089[0] 1776190278.189844 mapread 0x22d3a2 thru 0x239fc0 (0xcc1f bytes) 000090[0] 1776190278.196556 trunc from 0x4e4853 to 0x1da1dd (0x30a675 bytes) truncating to largest ever: 0x9bcbf9 000091[1] 1776190278.256594 trunc from 0x1da1de to 0x9bcbf8 (0x7e2a1b bytes) 000092[0] 1776190278.331735 mapread 0x32837a thru 0x32fc9a (0x7921 bytes) 000094[0] 1776190278.404161 write 0x3a8343 thru 0x3b587b (0xd539 bytes) 000096[0] 1776190278.551160 mapwrite 0x8f0d3a thru 0x8fb5f3 (0xa8ba bytes) 000097[1] 1776190278.866493 trunc from 0x9bcbf9 to 0x373d2e (0x648eca bytes) 000100[1] 1776190278.991252 trunc from 0x373d2f to 0x1df684 (0x1946aa bytes) 000101[1] 1776190279.081747 mapwrite 0x4277ea thru 0x4312f4 (0x9b0b bytes) 000103[0] 1776190279.359334 trunc from 0x4312f5 to 0x8adeeb (0x47cbf7 bytes) 000104[1] 1776190279.482247 read 0x4e7688 thru 0x4e8225 (0xb9e bytes) 000105[1] 1776190279.553640 mapwrite 0x92c31c thru 0x93af66 (0xec4b bytes) 000106[0] 1776190279.992838 mapwrite 0x1f661c thru 0x20077a (0xa15f bytes) 000109[1] 1776190280.277101 read 0x4cf1f6 thru 0x4de988 (0xf793 bytes) 000111[0] 1776190280.360842 trunc from 0x93af67 to 0x084129 (0x8b6e3d bytes) 000112[1] 1776190280.423525 write 0x44340e thru 0x446989 (0x357c bytes) 000113[1] 1776190280.484831 write 0x5f3620 thru 0x603028 (0xfa09 bytes) 000116[1] 1776190280.557848 read 0x4403ba thru 0x448348 (0x7f8f bytes) 000117[1] 1776190280.582070 mapread 0x09ac8e thru 0x0a7cd6 (0xd049 bytes) 000118[0] 1776190280.659267 write 0x044773 thru 0x052295 (0xdb23 bytes) 000120[1] 1776190280.718833 mapwrite 0x86edc2 thru 0x87d9fa (0xec39 bytes) 000123[0] 1776190281.190668 mapwrite 0x6e4ee9 thru 0x6f1ce9 (0xce01 bytes) 000125[0] 1776190281.418949 mapwrite 0x3bc52f thru 0x3c1610 (0x50e2 bytes) 000127[1] 1776190281.593394 trunc from 0x87d9fb to 0x435ddf (0x447c1b bytes) 000128[1] 1776190281.638147 mapwrite 0x5c576c thru 0x5ca0d4 (0x4969 bytes) 000133[0] 1776190281.756740 mapwrite 0x0c6061 thru 0x0c6eb8 (0xe58 bytes) 000134[1] 1776190281.874331 mapread 0x0003be thru 0x00b9ca (0xb60d bytes) 000137[0] 1776190281.990392 mapread 0x2bc51a thru 0x2c5f73 (0x9a5a bytes) 000138[1] 1776190282.060777 mapread 0x169556 thru 0x16f3e4 (0x5e8f bytes) 000139[0] 1776190282.115136 write 0x8b0a56 thru 0x8b49da (0x3f85 bytes) 000140[1] 1776190282.142150 trunc from 0x8b49db to 0x656cbc (0x25dd1e bytes) 000141[1] 1776190282.178729 mapread 0x44e03b thru 0x457a75 (0x9a3b bytes) 000144[1] 1776190282.220543 mapwrite 0x9ecd35 thru 0x9f40ee (0x73ba bytes) 000150[0] 1776190282.384871 mapwrite 0x216fbf thru 0x21d4e6 (0x6528 bytes) 000155[0] 1776190282.659876 mapread 0x16d3c1 thru 0x178cca (0xb90a bytes) 000159[0] 1776190282.874928 write 0x9db158 thru 0x9db4ab (0x354 bytes) 000161[1] 1776190282.975498 trunc from 0x9f40ef to 0x24f6dc (0x7a4a12 bytes) 000162[1] 1776190283.096935 mapwrite 0x31a133 thru 0x31b6fa (0x15c8 bytes) 000163[1] 1776190283.260739 mapwrite 0x031d5e thru 0x03a896 (0x8b39 bytes) 000167[0] 1776190283.734584 mapread 0x2c9471 thru 0x2d4ce9 (0xb879 bytes) 000168[0] 1776190283.892517 write 0x75575c thru 0x75f405 (0x9caa bytes) 000169[0] 1776190283.940340 trunc from 0x75f406 to 0x688710 (0xd6cf5 bytes) 000172[0] 1776190284.086804 trunc from 0x688711 to 0x52da5f (0x15acb1 bytes) 000175[1] 1776190284.122979 trunc from 0x52da60 to 0x3fb33f (0x132720 bytes) 000177[0] 1776190284.237447 mapwrite 0x235eaa thru 0x23ffe2 (0xa139 bytes) 000178[1] 1776190284.551938 mapwrite 0x2baf82 thru 0x2c25f3 (0x7672 bytes) 000180[1] 1776190284.779326 mapread 0x2431ac thru 0x24c932 (0x9787 bytes) 000181[0] 1776190284.831889 mapwrite 0x5d92aa thru 0x5e05bd (0x7314 bytes) 000182[0] 1776190285.018059 read 0x4d868c thru 0x4e2adc (0xa451 bytes) 000183[0] 1776190285.107982 trunc from 0x5e05be to 0x418a84 (0x1c7b39 bytes) 000184[1] 1776190285.132553 mapread 0x22082b thru 0x2252cd (0x4aa3 bytes) 000185[1] 1776190285.231637 write 0x967c3e thru 0x976033 (0xe3f6 bytes) 000186[0] 1776190285.250051 read 0x2b4cc1 thru 0x2ba4a0 (0x57e0 bytes) 000187[0] 1776190285.273823 read 0x0a837e thru 0x0b42e4 (0xbf67 bytes) 000189[0] 1776190285.323981 write 0x415ea7 thru 0x424cc8 (0xee22 bytes) 000190[0] 1776190285.365495 write 0x82bde9 thru 0x8309b8 (0x4bd0 bytes) 000191[0] 1776190285.384245 mapread 0x4eb9df thru 0x4fa092 (0xe6b4 bytes) 000192[1] 1776190285.560452 mapwrite 0x0b453f thru 0x0c0390 (0xbe52 bytes) 000195[0] 1776190285.800687 mapwrite 0x7dc315 thru 0x7e0bfc (0x48e8 bytes) 000196[1] 1776190285.988571 mapwrite 0x3dd32c thru 0x3dd683 (0x358 bytes) 000197[1] 1776190286.097182 write 0x7ff88b thru 0x808055 (0x87cb bytes) 000198[1] 1776190286.158323 write 0x943f02 thru 0x94b1b7 (0x72b6 bytes) 000204[0] 1776190286.241847 read 0x7ad6dc thru 0x7b6b13 (0x9438 bytes) 000207[0] 1776190286.382638 mapread 0x03d3f2 thru 0x04408d (0x6c9c bytes) 000208[1] 1776190286.589185 mapwrite 0x61818b thru 0x61af79 (0x2def bytes) 000209[0] 1776190286.713104 trunc from 0x976034 to 0x244811 (0x731822 bytes) 000212[0] 1776190286.859924 read 0x07845d thru 0x0837a7 (0xb34b bytes) 000213[0] 1776190287.034441 read 0x0d051b thru 0x0df92d (0xf413 bytes) 000218[1] 1776190287.202147 write 0x772ed3 thru 0x773480 (0x5ae bytes) 000221[1] 1776190287.289905 mapread 0x1c1816 thru 0x1ce798 (0xcf83 bytes) 000222[1] 1776190287.397482 write 0x940051 thru 0x942b7d (0x2b2d bytes) 000226[0] 1776190287.491589 mapread 0x44ff69 thru 0x451892 (0x192a bytes) 000227[1] 1776190287.563512 mapread 0x281afe thru 0x28ac87 (0x918a bytes) 000228[1] 1776190287.626635 write 0x5a5789 thru 0x5ad983 (0x81fb bytes) 000229[1] 1776190287.672931 mapread 0x6e0723 thru 0x6e9b84 (0x9462 bytes) 000231[1] 1776190287.720441 read 0x8649cd thru 0x870575 (0xbba9 bytes) 000236[0] 1776190287.878570 read 0x5ef2f9 thru 0x5f04dc (0x11e4 bytes) 000237[0] 1776190287.984192 read 0x710826 thru 0x71d869 (0xd044 bytes) 000238[0] 1776190287.989884 mapread 0x24d87f thru 0x25bb2e (0xe2b0 bytes) 000239[0] 1776190288.104945 mapread 0x25f484 thru 0x26e54e (0xf0cb bytes) 000240[0] 1776190288.291568 trunc from 0x942b7e to 0x0132d5 (0x92f8a8 bytes) 000241[1] 1776190288.407462 mapwrite 0x3836f7 thru 0x38ff6e (0xc878 bytes) 000247[1] 1776190288.701457 mapwrite 0x86eae1 thru 0x873d8a (0x52aa bytes) 000250[0] 1776190288.878676 mapwrite 0x2551aa thru 0x256bfe (0x1a55 bytes) 000252[1] 1776190289.058705 write 0x286814 thru 0x28b46f (0x4c5c bytes) 000253[1] 1776190289.143548 write 0x818e97 thru 0x8239b6 (0xab20 bytes) 000254[0] 1776190289.160347 write 0x142caf thru 0x14a560 (0x78b2 bytes) 000255[1] 1776190289.219441 trunc from 0x873d8b to 0x604a0b (0x26f37f bytes) 000262[0] 1776190289.575670 mapwrite 0x74b77f thru 0x75ad9c (0xf61e bytes) 000265[1] 1776190289.798554 mapwrite 0x843261 thru 0x84ab53 (0x78f3 bytes) 000267[1] 1776190289.969691 trunc from 0x84ab54 to 0x410d3a (0x439e19 bytes) 000270[0] 1776190290.017570 read 0x18584a thru 0x18b147 (0x58fe bytes) 000273[1] 1776190290.078724 read 0x18e015 thru 0x1921f2 (0x41de bytes) 000274[1] 1776190290.116128 mapwrite 0x1f95cd thru 0x1fd138 (0x3b6c bytes) 000275[0] 1776190290.267123 mapread 0x2ab220 thru 0x2b537f (0xa160 bytes) 000277[0] 1776190290.341226 read 0x0ea49b thru 0x0f05d5 (0x613b bytes) 000279[1] 1776190290.394160 read 0x101f06 thru 0x103cea (0x1de5 bytes) 000282[1] 1776190290.490386 write 0x905429 thru 0x90e9bb (0x9593 bytes) 000283[1] 1776190290.517709 write 0x992c86 thru 0x99a72d (0x7aa8 bytes) 000284[0] 1776190290.519443 read 0x602644 thru 0x606227 (0x3be4 bytes) 000286[0] 1776190290.577122 trunc from 0x99a72e to 0x2117e1 (0x788f4c bytes) 000287[0] 1776190290.651401 write 0x8b4c09 thru 0x8bd74f (0x8b47 bytes) 000288[0] 1776190290.669147 mapwrite 0x1b6649 thru 0x1b8d14 (0x26cc bytes) 000289[0] 1776190290.770424 mapread 0x20546a thru 0x21136f (0xbf06 bytes) 000297[0] 1776190290.917509 write 0x3cc2cf thru 0x3d2ba8 (0x68da bytes) 000299[1] 1776190290.962394 mapread 0x817673 thru 0x825b4e (0xe4dc bytes) 000300[1] 1776190291.184148 mapread 0x7577cd thru 0x75e407 (0x6c3b bytes) 000304[1] 1776190291.291384 write 0x29b0da thru 0x2a80b8 (0xcfdf bytes) 000306[1] 1776190291.328574 read 0x583dab thru 0x583f99 (0x1ef bytes) 000307[0] 1776190291.400257 write 0x256143 thru 0x2598fa (0x37b8 bytes) 000308[1] 1776190291.439876 mapread 0x3fe1ba thru 0x40d403 (0xf24a bytes) 000309[0] 1776190291.627955 trunc from 0x8bd750 to 0x26b9cb (0x651d84 bytes) 000310[1] 1776190291.732934 read 0x1bbb97 thru 0x1c72ad (0xb717 bytes) 000313[1] 1776190291.772509 mapread 0x0e4c78 thru 0x0efa69 (0xadf2 bytes) 000314[1] 1776190291.883492 read 0x10b23d thru 0x117c65 (0xca29 bytes) 000317[0] 1776190291.911893 read 0x0e19d0 thru 0x0eb76b (0x9d9c bytes) 000318[1] 1776190291.958051 write 0x1a9929 thru 0x1b97d0 (0xfea8 bytes) 000319[1] 1776190291.993501 trunc from 0x26b9cc to 0x4e1e9d (0x2764d2 bytes) 000320[0] 1776190292.053299 write 0x44e9ee thru 0x4527ef (0x3e02 bytes) 000321[1] 1776190292.134342 mapwrite 0x75987f thru 0x75ff32 (0x66b4 bytes) 000324[0] 1776190292.499413 mapwrite 0x730359 thru 0x73f0c1 (0xed69 bytes) 000327[0] 1776190292.832096 write 0x199b48 thru 0x19cb4a (0x3003 bytes) 000329[0] 1776190292.926272 write 0x532520 thru 0x534a7d (0x255e bytes) 000332[1] 1776190293.034461 mapread 0x18e361 thru 0x18ff8d (0x1c2d bytes) 000333[1] 1776190293.080607 write 0x58f567 thru 0x599845 (0xa2df bytes) 000334[0] 1776190293.122401 mapread 0x2522f1 thru 0x25fe92 (0xdba2 bytes) 000335[1] 1776190293.276221 mapwrite 0x9d07b2 thru 0x9d671a (0x5f69 bytes) 000336[1] 1776190293.410859 mapread 0x5a36a5 thru 0x5b1df8 (0xe754 bytes) 000337[0] 1776190293.588523 write 0x55892b thru 0x55ae68 (0x253e bytes) 000339[0] 1776190293.688250 mapwrite 0x4d6d7a thru 0x4e2f8f (0xc216 bytes) 000343[0] 1776190293.980112 write 0x7bac1c thru 0x7c697b (0xbd60 bytes) 000344[0] 1776190294.006173 mapwrite 0x9cfadd thru 0x9d275c (0x2c80 bytes) 000345[1] 1776190294.159166 trunc from 0x9d671b to 0x8d321f (0x1034fb bytes) 000348[0] 1776190294.355492 trunc from 0x8d3220 to 0x10609d (0x7cd182 bytes) 000350[0] 1776190294.414445 mapwrite 0x7b6eed thru 0x7b75bc (0x6d0 bytes) 000355[1] 1776190294.887991 read 0x3a9153 thru 0x3aae16 (0x1cc4 bytes) 000356[0] 1776190294.971330 trunc from 0x7b75bd to 0x3a23a1 (0x41521b bytes) 000358[1] 1776190295.070089 mapwrite 0x866f18 thru 0x87176c (0xa855 bytes) 000359[1] 1776190295.297806 read 0x3c394b thru 0x3d0db7 (0xd46d bytes) 000361[0] 1776190295.365496 mapwrite 0x59941c thru 0x59f768 (0x634d bytes) 000362[0] 1776190295.722412 trunc from 0x87176d to 0x7d797f (0x99ded bytes) 000363[1] 1776190295.838482 mapread 0x38d825 thru 0x393dc8 (0x65a4 bytes) 000364[0] 1776190295.994312 read 0x5d1d5a thru 0x5dd638 (0xb8df bytes) 000368[0] 1776190296.083947 read 0x469bd8 thru 0x46ba10 (0x1e39 bytes) 000369[0] 1776190296.129270 write 0x371794 thru 0x377721 (0x5f8e bytes) 000370[1] 1776190296.170346 write 0x17ece4 thru 0x180db3 (0x20d0 bytes) 000371[0] 1776190296.213556 trunc from 0x7d7980 to 0x19ecb6 (0x638cc9 bytes) 000372[1] 1776190296.279358 read 0x038e86 thru 0x040a8b (0x7c06 bytes) 000374[0] 1776190296.342412 trunc from 0x19ecb7 to 0x6581a8 (0x4b94f2 bytes) 000378[1] 1776190296.436885 read 0x37dca2 thru 0x388d0f (0xb06e bytes) 000379[0] 1776190296.468348 write 0x2242f1 thru 0x224e21 (0xb31 bytes) 000381[0] 1776190296.485723 read 0x5e12c6 thru 0x5e1fde (0xd19 bytes) 000382[1] 1776190296.501191 trunc from 0x6581a9 to 0x6256c5 (0x32ae3 bytes) 000384[0] 1776190296.607267 mapwrite 0x0741fc thru 0x0832e8 (0xf0ed bytes) 000385[0] 1776190296.786335 read 0x2bef8a thru 0x2cda64 (0xeadb bytes) 000386[0] 1776190296.852703 mapread 0x5313dd thru 0x53e97f (0xd5a3 bytes) 000387[0] 1776190297.151143 mapread 0x4bc3f1 thru 0x4cc115 (0xfd25 bytes) 000388[0] 1776190297.191491 trunc from 0x6256c6 to 0x44de86 (0x1d783f bytes) 000392[0] 1776190297.389130 mapread 0x2d460d thru 0x2da95d (0x6351 bytes) 000393[1] 1776190297.390484 write 0x41a9f5 thru 0x423c42 (0x924e bytes) 000394[0] 1776190297.464300 write 0x7e2329 thru 0x7e484d (0x2525 bytes) 000396[0] 1776190297.522144 write 0x109b79 thru 0x11403a (0xa4c2 bytes) 000402[0] 1776190297.711649 trunc from 0x7e484e to 0x8a14d0 (0xbcc83 bytes) 000404[1] 1776190297.756957 mapread 0x2cf8f3 thru 0x2d3d49 (0x4457 bytes) 000405[1] 1776190297.830084 read 0x731acb thru 0x73334f (0x1885 bytes) 000406[1] 1776190297.899135 write 0x5489fe thru 0x54ee28 (0x642b bytes) 000408[0] 1776190297.930135 mapwrite 0x1f537d thru 0x1ff8df (0xa563 bytes) 000411[0] 1776190298.135947 write 0x308dcf thru 0x30a862 (0x1a94 bytes) 000412[0] 1776190298.161860 mapwrite 0x51d34b thru 0x52c837 (0xf4ed bytes) 000419[1] 1776190298.497733 mapwrite 0x89134a thru 0x89cfce (0xbc85 bytes) 000420[1] 1776190298.880729 read 0x503a6b thru 0x50cb48 (0x90de bytes) 000421[0] 1776190298.967862 mapwrite 0x8bb0ef thru 0x8c972b (0xe63d bytes) 000424[1] 1776190299.225460 mapread 0x70f0a9 thru 0x71ec9e (0xfbf6 bytes) 000426[1] 1776190299.368077 trunc from 0x8c972c to 0x450475 (0x4792b6 bytes) 000427[0] 1776190299.442193 write 0x767018 thru 0x7697d5 (0x27be bytes) 000428[1] 1776190299.495249 mapread 0x0e7830 thru 0x0e9e57 (0x2628 bytes) 000429[1] 1776190299.563120 trunc from 0x7697d6 to 0x704dd4 (0x64a01 bytes) 000430[0] 1776190299.635462 mapwrite 0x7deb96 thru 0x7ee137 (0xf5a2 bytes) 000431[1] 1776190299.893373 mapwrite 0x6613db thru 0x666124 (0x4d4a bytes) 000432[0] 1776190300.031614 trunc from 0x7ee138 to 0x7dbdab (0x1238c bytes) 000433[1] 1776190300.096818 trunc from 0x7dbdac to 0x054e1b (0x786f90 bytes) 000434[0] 1776190300.165719 trunc from 0x054e1c to 0x193d4b (0x13ef30 bytes) 000435[0] 1776190300.232635 trunc from 0x193d4c to 0x12d56d (0x667de bytes) 000437[1] 1776190300.271943 write 0x430063 thru 0x43ddee (0xdd8c bytes) 000438[0] 1776190300.308121 trunc from 0x43ddef to 0x029e9e (0x413f50 bytes) 000439[0] 1776190300.393716 write 0x853749 thru 0x85d732 (0x9fea bytes) 000440[0] 1776190300.408994 read 0x5802f8 thru 0x58248c (0x2195 bytes) 000441[0] 1776190300.482975 mapwrite 0x4af0ef thru 0x4b7ef1 (0x8e03 bytes) 000446[0] 1776190300.736468 trunc from 0x85d733 to 0x8d63ef (0x78cbd bytes) 000448[0] 1776190300.790102 trunc from 0x8d63f0 to 0x07361b (0x862dd4 bytes) 000449[0] 1776190300.817554 mapwrite 0x7621c4 thru 0x762e13 (0xc50 bytes) 000451[0] 1776190300.943664 mapread 0x68cb04 thru 0x694b8a (0x8087 bytes) 000454[0] 1776190301.095561 trunc from 0x762e14 to 0x65a569 (0x1088aa bytes) 000455[0] 1776190301.133074 mapwrite 0x7efa8b thru 0x7f253e (0x2ab4 bytes) 000456[0] 1776190301.300989 read 0x25902e thru 0x25d59c (0x456f bytes) 000457[0] 1776190301.391650 trunc from 0x7f253f to 0x1dd3ad (0x615191 bytes) 000458[1] 1776190301.465258 mapwrite 0x18090e thru 0x18d43a (0xcb2d bytes) 000461[0] 1776190301.848685 write 0x074e9f thru 0x07f3eb (0xa54d bytes) 000464[0] 1776190301.909303 trunc from 0x1dd3ae to 0x889a44 (0x6ac697 bytes) 000470[1] 1776190301.984379 mapwrite 0x70dd17 thru 0x711f8b (0x4275 bytes) 000472[1] 1776190302.098973 mapwrite 0x170e21 thru 0x17cc08 (0xbde8 bytes) 000473[0] 1776190302.271062 read 0x45168e thru 0x457398 (0x5d0b bytes) 000474[0] 1776190302.302523 read 0x212dc5 thru 0x21d4ae (0xa6ea bytes) 000477[0] 1776190302.427339 trunc from 0x889a45 to 0x481c8e (0x407db6 bytes) 000479[0] 1776190302.521371 mapwrite 0x877fcc thru 0x887668 (0xf69d bytes) 000485[1] 1776190302.753432 write 0x68fa6a thru 0x69345b (0x39f2 bytes) 000486[1] 1776190302.806009 trunc from 0x887669 to 0x56999a (0x31dcce bytes) 000487[1] 1776190302.853907 trunc from 0x56999b to 0x3f7ccd (0x171ccd bytes) 000488[1] 1776190302.901719 mapwrite 0x236f6b thru 0x2406c0 (0x9756 bytes) 000489[1] 1776190303.018907 read 0x332e52 thru 0x33ade7 (0x7f96 bytes) 000491[0] 1776190303.090524 read 0x2686bc thru 0x26a67f (0x1fc4 bytes) 000492[0] 1776190303.129503 mapwrite 0x570130 thru 0x57a738 (0xa609 bytes) 000497[0] 1776190303.366914 write 0x99671e thru 0x9a21ee (0xbad1 bytes) 000498[0] 1776190303.399932 mapwrite 0x18e3c7 thru 0x1952b7 (0x6ef1 bytes) 000500[1] 1776190303.526294 read 0x17a937 thru 0x185cdc (0xb3a6 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 303 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[0] 1776190304.289562 mapwrite 0x382000 thru 0x386abb (0x4abc bytes) truncating to largest ever: 0x910653 000003[0] 1776190304.473012 trunc from 0x386abc to 0x910652 (0x589b97 bytes) 000004[0] 1776190304.511561 read_OD 0x176000 thru 0x179fff (0x4000 bytes) 000005[1] 1776190304.530010 read 0x87f000 thru 0x88ca04 (0xda05 bytes) 000006[1] 1776190304.546487 mapread 0x260000 thru 0x2650ec (0x50ed bytes) 000007[1] 1776190304.566600 trunc from 0x910653 to 0x0f1c33 (0x81ea1f bytes) 000008[1] 1776190304.596154 mapwrite 0x78d000 thru 0x796e4b (0x9e4c bytes) 000009[1] 1776190304.694089 write 0x087000 thru 0x093f36 (0xcf37 bytes) 000010[1] 1776190304.699219 mapwrite 0x09e000 thru 0x0aabcb (0xcbcc bytes) 000012[1] 1776190304.865204 mapwrite 0x4b3000 thru 0x4b5cee (0x2cef bytes) 000013[1] 1776190304.961784 trunc from 0x796e4c to 0x299dc8 (0x4fd083 bytes) 000016[1] 1776190305.008803 trunc from 0x299dc9 to 0x354830 (0xbaa68 bytes) 000017[0] 1776190305.044759 mapwrite 0x18c000 thru 0x19b632 (0xf633 bytes) 000018[1] 1776190305.158004 trunc from 0x354831 to 0x7fa736 (0x4a5f06 bytes) 000023[0] 1776190305.265746 read_OD 0x3c8000 thru 0x3cafff (0x3000 bytes) 000024[0] 1776190305.288327 read_OD 0x748000 thru 0x754fff (0xd000 bytes) 000026[1] 1776190305.298885 mapwrite 0x1e4000 thru 0x1ef00a (0xb00b bytes) 000029[0] 1776190305.463776 read_OD 0x3d3000 thru 0x3d7fff (0x5000 bytes) 000031[0] 1776190305.491272 write_OD 0x019000 thru 0x021fff (0x9000 bytes) 000032[1] 1776190305.541199 write 0x9d7000 thru 0x9e6250 (0xf251 bytes) 000033[1] 1776190305.551706 trunc from 0x9e6251 to 0x6a3f42 (0x34230e bytes) 000035[0] 1776190305.621856 mapread 0x3c2000 thru 0x3cca12 (0xaa13 bytes) 000038[1] 1776190305.877670 read 0x220000 thru 0x2255f7 (0x55f8 bytes) 000039[0] 1776190305.966333 trunc from 0x6a3f43 to 0x85a709 (0x1b67c7 bytes) 000041[1] 1776190306.098065 write 0x152000 thru 0x159c09 (0x7c0a bytes) 000043[1] 1776190306.215993 mapread 0x1c2000 thru 0x1cd398 (0xb399 bytes) 000045[0] 1776190306.368102 trunc from 0x85a70a to 0x8e796f (0x8d266 bytes) 000047[1] 1776190306.446704 mapread 0x388000 thru 0x389f9a (0x1f9b bytes) 000049[1] 1776190306.488729 read 0x4fd000 thru 0x4ffbc6 (0x2bc7 bytes) 000050[1] 1776190306.553121 read 0x740000 thru 0x749490 (0x9491 bytes) 000052[0] 1776190306.564757 mapread 0x5a5000 thru 0x5aae9c (0x5e9d bytes) 000053[0] 1776190306.648605 mapwrite 0x025000 thru 0x032508 (0xd509 bytes) 000054[0] 1776190306.836981 read_OD 0x638000 thru 0x645fff (0xe000 bytes) 000055[0] 1776190306.861431 write_OD 0x4d3000 thru 0x4e0fff (0xe000 bytes) 000056[0] 1776190306.938949 read_OD 0x084000 thru 0x084fff (0x1000 bytes) 000057[1] 1776190306.949994 write 0x5ed000 thru 0x5f3c48 (0x6c49 bytes) 000059[1] 1776190306.999709 mapread 0x7af000 thru 0x7b0e2f (0x1e30 bytes) 000060[0] 1776190307.043093 mapwrite 0x327000 thru 0x329a74 (0x2a75 bytes) 000061[0] 1776190307.300848 read_OD 0x613000 thru 0x61ffff (0xd000 bytes) 000062[0] 1776190307.333546 mapread 0x2a6000 thru 0x2b1c41 (0xbc42 bytes) 000063[1] 1776190307.450320 mapwrite 0x1e3000 thru 0x1e7fd0 (0x4fd1 bytes) 000064[0] 1776190307.562870 write_OD 0x19c000 thru 0x19ffff (0x4000 bytes) 000069[0] 1776190307.615738 read_OD 0x479000 thru 0x47cfff (0x4000 bytes) 000071[0] 1776190307.621068 read_OD 0x14b000 thru 0x150fff (0x6000 bytes) truncating to largest ever: 0x94f4df 000072[0] 1776190307.628140 trunc from 0x8e7970 to 0x94f4de (0x67b6f bytes) 000073[1] 1776190307.661944 mapwrite 0x372000 thru 0x37317a (0x117b bytes) 000075[1] 1776190307.752278 trunc from 0x94f4df to 0x80f5b3 (0x13ff2b bytes) 000076[1] 1776190307.796903 trunc from 0x80f5b4 to 0x752526 (0xbd08d bytes) 000077[1] 1776190307.833660 trunc from 0x752527 to 0x4fb524 (0x257002 bytes) 000081[0] 1776190307.869642 write_OD 0x0af000 thru 0x0bbfff (0xd000 bytes) 000085[1] 1776190307.931959 write 0x03f000 thru 0x04bb03 (0xcb04 bytes) 000086[0] 1776190307.966933 mapread 0x18a000 thru 0x196976 (0xc977 bytes) 000087[1] 1776190308.043688 write 0x7c9000 thru 0x7d0624 (0x7625 bytes) 000088[1] 1776190308.067456 mapread 0x506000 thru 0x50cede (0x6edf bytes) 000089[1] 1776190308.103133 write 0x5fd000 thru 0x60c448 (0xf449 bytes) 000090[1] 1776190308.124156 mapread 0x729000 thru 0x73649a (0xd49b bytes) 000091[1] 1776190308.214946 mapwrite 0x7f4000 thru 0x7f7b1c (0x3b1d bytes) 000093[1] 1776190308.375760 read 0x700000 thru 0x705528 (0x5529 bytes) 000094[1] 1776190308.440166 write 0x6e5000 thru 0x6eee94 (0x9e95 bytes) 000096[1] 1776190308.480867 read 0x7f7000 thru 0x7f7875 (0x876 bytes) 000102[0] 1776190308.528611 mapread 0x468000 thru 0x474c83 (0xcc84 bytes) 000105[1] 1776190308.653277 mapwrite 0x38b000 thru 0x39a711 (0xf712 bytes) 000106[0] 1776190308.836971 read_OD 0x548000 thru 0x54dfff (0x6000 bytes) 000108[0] 1776190308.894375 mapread 0x649000 thru 0x64eaff (0x5b00 bytes) 000109[1] 1776190308.969112 mapwrite 0x7f0000 thru 0x7fe262 (0xe263 bytes) 000117[1] 1776190309.254513 mapread 0x433000 thru 0x43ab23 (0x7b24 bytes) 000118[0] 1776190309.339757 read_OD 0x18f000 thru 0x199fff (0xb000 bytes) 000121[0] 1776190309.391359 read_OD 0x1fb000 thru 0x208fff (0xe000 bytes) 000122[1] 1776190309.404958 write 0x40b000 thru 0x418cc7 (0xdcc8 bytes) 000123[0] 1776190309.430461 read_OD 0x147000 thru 0x151fff (0xb000 bytes) 000126[0] 1776190309.474469 mapwrite 0x55e000 thru 0x56810a (0xa10b bytes) 000129[0] 1776190309.801124 trunc from 0x7fe263 to 0x594ba1 (0x2696c1 bytes) 000131[1] 1776190309.835985 read 0x16d000 thru 0x16e8e0 (0x18e1 bytes) 000132[0] 1776190309.967834 mapwrite 0x5a9000 thru 0x5ad7c8 (0x47c9 bytes) 000135[0] 1776190310.104813 write_OD 0x6e4000 thru 0x6f2fff (0xf000 bytes) 000136[0] 1776190310.154729 mapwrite 0x1a0000 thru 0x1ac57d (0xc57e bytes) 000137[1] 1776190310.297109 write 0x4d4000 thru 0x4d56f1 (0x16f2 bytes) 000138[1] 1776190310.346543 trunc from 0x6f3000 to 0x2d91a0 (0x419e5f bytes) 000141[0] 1776190310.373002 mapwrite 0x2d1000 thru 0x2dc8ae (0xb8af bytes) 000143[1] 1776190310.550252 trunc from 0x2dc8af to 0x8a5be5 (0x5c9337 bytes) 000146[1] 1776190310.605627 read 0x034000 thru 0x03d8da (0x98db bytes) 000147[1] 1776190310.772857 write 0x8f7000 thru 0x8fea86 (0x7a87 bytes) 000148[0] 1776190310.773911 trunc from 0x8fea87 to 0x940aa4 (0x4201e bytes) 000149[0] 1776190310.884396 mapwrite 0x030000 thru 0x03fff7 (0xfff8 bytes) 000151[0] 1776190311.303866 trunc from 0x940aa5 to 0x39bc5f (0x5a4e45 bytes) 000153[0] 1776190311.427649 trunc from 0x39bc60 to 0x80d491 (0x471832 bytes) 000154[0] 1776190311.511742 mapwrite 0x190000 thru 0x19974e (0x974f bytes) 000155[1] 1776190311.757010 mapwrite 0x9b3000 thru 0x9c2774 (0xf775 bytes) 000156[1] 1776190311.973010 read 0x78e000 thru 0x7910b6 (0x30b7 bytes) 000159[0] 1776190312.043519 write_OD 0x45e000 thru 0x468fff (0xb000 bytes) 000160[1] 1776190312.111774 mapwrite 0x0d2000 thru 0x0d3b55 (0x1b56 bytes) 000161[0] 1776190312.261932 write_OD 0x6b6000 thru 0x6b9fff (0x4000 bytes) 000163[0] 1776190312.328637 write_OD 0x5c9000 thru 0x5cdfff (0x5000 bytes) 000164[1] 1776190312.382663 mapread 0x4ff000 thru 0x503a11 (0x4a12 bytes) 000165[0] 1776190312.407415 mapwrite 0x00b000 thru 0x01754e (0xc54f bytes) 000168[1] 1776190312.558897 read 0x4fa000 thru 0x4fc9f9 (0x29fa bytes) 000169[1] 1776190312.654535 read 0x5a4000 thru 0x5a5650 (0x1651 bytes) 000170[0] 1776190312.666879 mapwrite 0x199000 thru 0x1a4a75 (0xba76 bytes) 000175[1] 1776190312.867858 write 0x35a000 thru 0x365933 (0xb934 bytes) 000178[0] 1776190312.956353 read_OD 0x1e0000 thru 0x1e9fff (0xa000 bytes) 000179[0] 1776190313.037859 trunc from 0x9c2775 to 0x82a66c (0x198108 bytes) 000180[0] 1776190313.132448 mapwrite 0x82f000 thru 0x8363e7 (0x73e8 bytes) 000181[1] 1776190313.347411 read 0x1e3000 thru 0x1f2ee1 (0xfee2 bytes) 000182[1] 1776190313.421730 mapwrite 0x8a0000 thru 0x8ad900 (0xd901 bytes) 000184[1] 1776190313.638910 read 0x285000 thru 0x28edec (0x9ded bytes) 000187[1] 1776190313.814937 read 0x043000 thru 0x0487c5 (0x57c6 bytes) 000191[1] 1776190313.944137 mapread 0x558000 thru 0x565173 (0xd174 bytes) 000193[1] 1776190313.977766 write 0x29f000 thru 0x2a2264 (0x3265 bytes) 000194[0] 1776190314.012953 read_OD 0x62b000 thru 0x633fff (0x9000 bytes) 000196[1] 1776190314.143227 trunc from 0x8ad901 to 0x2e025b (0x5cd6a5 bytes) 000198[1] 1776190314.261747 write 0x071000 thru 0x07627e (0x527f bytes) 000199[0] 1776190314.277744 mapread 0x26e000 thru 0x27bcd8 (0xdcd9 bytes) 000202[1] 1776190314.510806 read 0x086000 thru 0x09485f (0xe860 bytes) 000204[0] 1776190314.621354 trunc from 0x2e025c to 0x43204c (0x151df1 bytes) 000205[1] 1776190314.668562 mapwrite 0x5cb000 thru 0x5d1c1c (0x6c1d bytes) 000206[1] 1776190314.824404 mapwrite 0x9ac000 thru 0x9b2a6b (0x6a6c bytes) 000208[1] 1776190315.019101 trunc from 0x9b2a6c to 0x04faab (0x962fc0 bytes) 000212[1] 1776190315.075528 trunc from 0x04faac to 0x535af4 (0x4e6049 bytes) 000213[1] 1776190315.101819 trunc from 0x535af5 to 0x75073c (0x21ac48 bytes) 000217[1] 1776190315.152405 mapread 0x37a000 thru 0x385614 (0xb615 bytes) 000221[1] 1776190315.266774 mapread 0x6b5000 thru 0x6b894e (0x394f bytes) 000223[0] 1776190315.313922 trunc from 0x75073d to 0x77f428 (0x2ecec bytes) 000227[1] 1776190315.389352 trunc from 0x77f429 to 0x39b0af (0x3e4379 bytes) 000230[1] 1776190315.495555 trunc from 0x39b0b0 to 0x226084 (0x17502b bytes) 000233[0] 1776190315.529928 write_OD 0x4ee000 thru 0x4f2fff (0x5000 bytes) 000234[1] 1776190315.669725 trunc from 0x4f3000 to 0x2c9e5c (0x2291a3 bytes) 000235[0] 1776190315.752572 write_OD 0x27b000 thru 0x283fff (0x9000 bytes) 000236[0] 1776190315.880960 mapread 0x08d000 thru 0x08e3fd (0x13fe bytes) 000238[0] 1776190315.924676 mapread 0x144000 thru 0x14e849 (0xa84a bytes) 000241[1] 1776190316.027894 write 0x328000 thru 0x32ad6c (0x2d6d bytes) 000242[1] 1776190316.058647 mapread 0x0a6000 thru 0x0b2af7 (0xcaf8 bytes) 000243[1] 1776190316.162778 mapwrite 0x751000 thru 0x752a74 (0x1a75 bytes) 000244[0] 1776190316.269523 read_OD 0x563000 thru 0x56cfff (0xa000 bytes) 000246[1] 1776190316.303624 mapread 0x426000 thru 0x4339ed (0xd9ee bytes) 000247[1] 1776190316.393755 mapwrite 0x404000 thru 0x4062db (0x22dc bytes) 000248[1] 1776190316.515000 read 0x4a7000 thru 0x4af8a7 (0x88a8 bytes) 000249[1] 1776190316.567605 write 0x3a2000 thru 0x3a71bb (0x51bc bytes) 000250[0] 1776190316.575649 read_OD 0x629000 thru 0x62afff (0x2000 bytes) 000251[0] 1776190316.605490 trunc from 0x752a75 to 0x6cfae9 (0x82f8b bytes) 000252[0] 1776190316.667511 read_OD 0x1b7000 thru 0x1bbfff (0x5000 bytes) 000255[1] 1776190316.700709 trunc from 0x6cfaea to 0x7b0323 (0xe083a bytes) 000261[1] 1776190316.831071 read 0x086000 thru 0x08d228 (0x7229 bytes) 000264[1] 1776190317.055755 read 0x560000 thru 0x5694d7 (0x94d8 bytes) 000265[0] 1776190317.072760 trunc from 0x7b0324 to 0x1ff989 (0x5b099a bytes) 000266[0] 1776190317.147392 trunc from 0x1ff98a to 0x41b2ae (0x21b925 bytes) 000267[1] 1776190317.173360 trunc from 0x41b2af to 0x7b3e27 (0x398b79 bytes) 000268[1] 1776190317.239718 mapread 0x1d4000 thru 0x1e3cde (0xfcdf bytes) 000269[1] 1776190317.458825 mapread 0x314000 thru 0x31898c (0x498d bytes) 000270[1] 1776190317.499392 trunc from 0x7b3e28 to 0x1d5678 (0x5de7af bytes) 000272[1] 1776190317.528397 write 0x5e9000 thru 0x5ec04b (0x304c bytes) 000273[0] 1776190317.564463 write_OD 0x747000 thru 0x748fff (0x2000 bytes) 000275[1] 1776190317.642273 mapwrite 0x924000 thru 0x92fbd2 (0xbbd3 bytes) 000276[0] 1776190317.902207 write_OD 0x1e6000 thru 0x1e9fff (0x4000 bytes) 000277[1] 1776190318.038476 mapread 0x905000 thru 0x911fb7 (0xcfb8 bytes) 000278[1] 1776190318.191011 read 0x869000 thru 0x86f0a5 (0x60a6 bytes) 000279[0] 1776190318.229106 trunc from 0x92fbd3 to 0x191204 (0x79e9ce bytes) 000280[1] 1776190318.270602 read 0x0db000 thru 0x0e5337 (0xa338 bytes) 000281[0] 1776190318.334387 trunc from 0x191205 to 0x60d979 (0x47c775 bytes) 000282[1] 1776190318.389358 trunc from 0x60d97a to 0x913c66 (0x3062ed bytes) 000283[0] 1776190318.454565 read_OD 0x667000 thru 0x66afff (0x4000 bytes) 000284[0] 1776190318.482161 write_OD 0x2d2000 thru 0x2dbfff (0xa000 bytes) 000285[0] 1776190318.552891 read_OD 0x085000 thru 0x08bfff (0x7000 bytes) 000286[1] 1776190318.568840 mapwrite 0x04e000 thru 0x05630e (0x830f bytes) 000287[0] 1776190318.911402 trunc from 0x913c67 to 0x5a4d23 (0x36ef43 bytes) 000288[1] 1776190319.071561 read 0x2d3000 thru 0x2d35fe (0x5ff bytes) 000292[1] 1776190319.214595 mapread 0x55c000 thru 0x564da3 (0x8da4 bytes) 000293[0] 1776190319.215335 mapread 0x5a3000 thru 0x5a4ab3 (0x1ab4 bytes) 000294[1] 1776190319.349222 mapread 0x4ab000 thru 0x4b97f5 (0xe7f6 bytes) 000295[0] 1776190319.357911 mapread 0x1a6000 thru 0x1ac623 (0x6624 bytes) 000299[1] 1776190319.362878 read 0x0e2000 thru 0x0eec4e (0xcc4f bytes) 000300[1] 1776190319.417254 write 0x5f3000 thru 0x5fe9ea (0xb9eb bytes) 000301[0] 1776190319.489758 write_OD 0x361000 thru 0x36afff (0xa000 bytes) 000305[0] 1776190319.697689 read_OD 0x37c000 thru 0x380fff (0x5000 bytes) 000306[1] 1776190319.712695 write 0x28f000 thru 0x29ba9d (0xca9e bytes) 000308[1] 1776190319.772004 mapwrite 0x8c2000 thru 0x8cd992 (0xb993 bytes) 000310[1] 1776190319.957365 mapread 0x367000 thru 0x3761cb (0xf1cc bytes) 000311[0] 1776190320.058898 read_OD 0x3c5000 thru 0x3cafff (0x6000 bytes) truncating to largest ever: 0x95235d 000313[1] 1776190320.118781 trunc from 0x8cd993 to 0x95235c (0x849ca bytes) 000314[0] 1776190320.173907 read_OD 0x82a000 thru 0x836fff (0xd000 bytes) 000317[1] 1776190320.208933 mapwrite 0x319000 thru 0x3245bb (0xb5bc bytes) 000322[1] 1776190320.564794 read 0x046000 thru 0x0462d2 (0x2d3 bytes) 000324[1] 1776190320.792659 mapread 0x44d000 thru 0x451309 (0x430a bytes) 000326[1] 1776190320.812987 mapwrite 0x312000 thru 0x31822e (0x622f bytes) 000330[1] 1776190320.923952 mapwrite 0x2c5000 thru 0x2d413a (0xf13b bytes) 000337[0] 1776190321.121700 mapwrite 0x8d0000 thru 0x8d1c6e (0x1c6f bytes) 000338[0] 1776190321.365238 mapwrite 0x57b000 thru 0x58316e (0x816f bytes) 000341[1] 1776190321.634192 read 0x4c0000 thru 0x4c3425 (0x3426 bytes) 000344[0] 1776190321.838207 mapread 0x4e4000 thru 0x4ec99f (0x89a0 bytes) 000345[0] 1776190321.950640 write_OD 0x8ea000 thru 0x8eafff (0x1000 bytes) 000346[0] 1776190322.109734 write_OD 0x683000 thru 0x68cfff (0xa000 bytes) 000347[0] 1776190322.185259 write_OD 0x9b7000 thru 0x9bffff (0x9000 bytes) 000348[0] 1776190322.281716 mapwrite 0x6d3000 thru 0x6e061e (0xd61f bytes) 000349[0] 1776190322.488793 trunc from 0x9c0000 to 0x83f3e5 (0x180c1a bytes) 000351[0] 1776190322.531837 mapread 0x7fa000 thru 0x801730 (0x7731 bytes) 000352[1] 1776190322.574074 read 0x6f6000 thru 0x7016fa (0xb6fb bytes) 000353[0] 1776190322.626864 write_OD 0x9fd000 thru 0x9fefff (0x2000 bytes) 000356[1] 1776190322.712463 read 0x506000 thru 0x50ba14 (0x5a15 bytes) 000357[1] 1776190322.785333 write 0x6fe000 thru 0x70713a (0x913b bytes) 000358[0] 1776190322.840462 mapread 0x9c9000 thru 0x9d70c1 (0xe0c2 bytes) 000360[1] 1776190322.962198 mapread 0x714000 thru 0x71977d (0x577e bytes) 000362[0] 1776190322.990845 trunc from 0x9ff000 to 0x276f55 (0x7880aa bytes) 000365[0] 1776190323.093436 write_OD 0x806000 thru 0x806fff (0x1000 bytes) 000367[0] 1776190323.189142 mapwrite 0x782000 thru 0x785255 (0x3256 bytes) 000368[0] 1776190323.284225 read_OD 0x140000 thru 0x142fff (0x3000 bytes) 000369[0] 1776190323.299483 mapwrite 0x269000 thru 0x26bcd0 (0x2cd1 bytes) 000370[1] 1776190323.501365 read 0x129000 thru 0x12c180 (0x3181 bytes) 000371[1] 1776190323.603788 mapwrite 0x04a000 thru 0x051be2 (0x7be3 bytes) 000373[0] 1776190323.818370 mapread 0x3e4000 thru 0x3efdd6 (0xbdd7 bytes) 000374[1] 1776190323.943799 mapwrite 0x722000 thru 0x72f81a (0xd81b bytes) 000377[1] 1776190324.159495 trunc from 0x807000 to 0x56affe (0x29c001 bytes) 000378[0] 1776190324.219285 mapwrite 0x5cf000 thru 0x5d1f29 (0x2f2a bytes) 000380[0] 1776190324.326920 mapread 0x3a0000 thru 0x3a377c (0x377d bytes) 000381[0] 1776190324.378562 write_OD 0x1bf000 thru 0x1c6fff (0x8000 bytes) 000382[1] 1776190324.430646 mapwrite 0x5fa000 thru 0x5fc112 (0x2113 bytes) 000383[1] 1776190324.528183 write 0x433000 thru 0x43c08c (0x908d bytes) 000384[0] 1776190324.561439 read_OD 0x2dc000 thru 0x2e7fff (0xc000 bytes) 000387[0] 1776190324.611713 read_OD 0x278000 thru 0x27efff (0x7000 bytes) 000388[1] 1776190324.622539 read 0x2c6000 thru 0x2c87a2 (0x27a3 bytes) 000391[1] 1776190324.686567 mapread 0x5f7000 thru 0x5fb439 (0x443a bytes) 000392[1] 1776190324.698613 mapread 0x4bb000 thru 0x4c0230 (0x5231 bytes) 000393[1] 1776190324.699460 read 0x378000 thru 0x38699b (0xe99c bytes) 000396[1] 1776190324.701327 mapread 0x3e5000 thru 0x3edbf3 (0x8bf4 bytes) 000399[0] 1776190324.702348 mapwrite 0x49d000 thru 0x49f6b7 (0x26b8 bytes) 000401[1] 1776190324.868566 trunc from 0x5fc113 to 0x72162a (0x125518 bytes) 000402[0] 1776190324.946457 write_OD 0x215000 thru 0x219fff (0x5000 bytes) 000403[0] 1776190325.007664 write_OD 0x396000 thru 0x39cfff (0x7000 bytes) 000406[1] 1776190325.065752 read 0x462000 thru 0x4713e9 (0xf3ea bytes) 000407[1] 1776190325.138512 write 0x473000 thru 0x47517d (0x217e bytes) 000408[0] 1776190325.169451 trunc from 0x72162b to 0x7db1e5 (0xb9bbb bytes) 000410[1] 1776190325.241065 mapread 0x073000 thru 0x07d402 (0xa403 bytes) 000411[1] 1776190325.407790 trunc from 0x7db1e6 to 0x0eca81 (0x6ee764 bytes) 000412[1] 1776190325.516255 trunc from 0x0eca82 to 0x685e99 (0x599418 bytes) 000413[1] 1776190325.566340 trunc from 0x685e9a to 0x24448b (0x441a0e bytes) 000416[1] 1776190325.681300 trunc from 0x24448c to 0x1c2c5f (0x8182c bytes) 000417[0] 1776190325.703488 mapread 0x010000 thru 0x017551 (0x7552 bytes) 000420[1] 1776190325.822597 trunc from 0x1c2c60 to 0x8c6434 (0x7037d5 bytes) 000421[1] 1776190325.923681 mapread 0x612000 thru 0x61d5c4 (0xb5c5 bytes) 000422[0] 1776190326.026226 mapwrite 0x5cb000 thru 0x5cf4d5 (0x44d6 bytes) 000423[1] 1776190326.179550 mapwrite 0x376000 thru 0x383a3f (0xda40 bytes) 000424[1] 1776190326.338297 read 0x88f000 thru 0x89b300 (0xc301 bytes) 000425[0] 1776190326.354927 write_OD 0x212000 thru 0x212fff (0x1000 bytes) 000426[0] 1776190326.425085 mapread 0x50d000 thru 0x514483 (0x7484 bytes) 000427[0] 1776190326.498469 trunc from 0x8c6435 to 0x7b4368 (0x1120cc bytes) 000429[1] 1776190326.586245 trunc from 0x7b4369 to 0x575aba (0x23e8ae bytes) 000432[0] 1776190326.680123 read_OD 0x4b4000 thru 0x4befff (0xb000 bytes) 000433[1] 1776190326.769894 mapwrite 0x8a2000 thru 0x8b12e9 (0xf2ea bytes) 000438[0] 1776190327.088099 read_OD 0x838000 thru 0x840fff (0x9000 bytes) 000441[0] 1776190327.126568 mapwrite 0x603000 thru 0x60c4f4 (0x94f5 bytes) 000442[1] 1776190327.315337 write 0x1a8000 thru 0x1a964c (0x164d bytes) 000443[0] 1776190327.333493 trunc from 0x8b12ea to 0x5b936c (0x2f7f7d bytes) 000444[1] 1776190327.387197 trunc from 0x5b936d to 0x46e9fd (0x14a96f bytes) 000447[1] 1776190327.413506 read 0x30d000 thru 0x312a60 (0x5a61 bytes) 000448[1] 1776190327.480537 trunc from 0x46e9fe to 0x2ec153 (0x1828aa bytes) 000449[0] 1776190327.509902 mapread 0x2e9000 thru 0x2ebc9e (0x2c9f bytes) 000453[0] 1776190327.565156 write_OD 0x4af000 thru 0x4bcfff (0xe000 bytes) 000459[1] 1776190327.696994 read 0x1a0000 thru 0x1af819 (0xf81a bytes) 000460[0] 1776190327.815478 mapread 0x437000 thru 0x43e646 (0x7647 bytes) 000461[1] 1776190327.895547 write 0x9ae000 thru 0x9afd2c (0x1d2d bytes) 000465[0] 1776190328.000101 trunc from 0x9afd2d to 0x805b2b (0x1aa201 bytes) 000466[1] 1776190328.060212 read 0x09c000 thru 0x0a7d76 (0xbd77 bytes) 000467[1] 1776190328.232560 trunc from 0x805b2c to 0x7db058 (0x2aad3 bytes) 000468[1] 1776190328.327580 mapwrite 0x92c000 thru 0x931c4a (0x5c4b bytes) 000469[0] 1776190328.533632 write_OD 0x6e6000 thru 0x6f1fff (0xc000 bytes) 000470[0] 1776190328.602525 trunc from 0x931c4b to 0x5009b1 (0x431299 bytes) 000471[0] 1776190328.653150 mapwrite 0x98b000 thru 0x99a045 (0xf046 bytes) 000472[0] 1776190328.792563 read_OD 0x07d000 thru 0x082fff (0x6000 bytes) 000473[0] 1776190328.819775 write_OD 0x418000 thru 0x418fff (0x1000 bytes) 000474[1] 1776190328.876644 write 0x4a6000 thru 0x4ada0c (0x7a0d bytes) 000475[0] 1776190328.919258 trunc from 0x99a046 to 0x00273f (0x997906 bytes) 000476[1] 1776190328.984199 mapread 00000000 thru 0x001765 (0x1766 bytes) 000481[1] 1776190329.020206 mapwrite 0x104000 thru 0x10ef48 (0xaf49 bytes) 000482[1] 1776190329.229858 read 0x009000 thru 0x011dfc (0x8dfd bytes) 000483[1] 1776190329.304136 write 0x65f000 thru 0x6689cd (0x99ce bytes) 000484[0] 1776190329.311561 trunc from 0x6689ce to 0x7bbe2b (0x15345e bytes) 000486[1] 1776190329.399392 mapread 0x446000 thru 0x455e5b (0xfe5c bytes) 000487[0] 1776190329.519940 trunc from 0x7bbe2c to 0x8902c6 (0xd449b bytes) 000489[0] 1776190329.603749 write_OD 0x695000 thru 0x69ffff (0xb000 bytes) 000491[0] 1776190329.656504 mapwrite 0x72f000 thru 0x73ec9e (0xfc9f bytes) 000492[1] 1776190329.871889 write 0x6e4000 thru 0x6e5717 (0x1718 bytes) 000493[1] 1776190329.954596 trunc from 0x8902c7 to 0x36dadc (0x5227ea bytes) 000494[1] 1776190329.984871 mapread 0x052000 thru 0x05e501 (0xc502 bytes) 000495[1] 1776190330.120089 mapread 0x0d9000 thru 0x0daa6e (0x1a6f bytes) 000498[0] 1776190330.188878 mapwrite 0x861000 thru 0x86279e (0x179f bytes) 000499[0] 1776190330.356312 mapwrite 0x47d000 thru 0x4834d8 (0x64d9 bytes) 000500[0] 1776190330.510133 mapwrite 0x14c000 thru 0x156b30 (0xab31 bytes) All operations completed A-OK! PASS 16a (66s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 14:12:15 (1776190335) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.340034 s, 6.2 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.289187 s, 7.3 MB/s Chance of close/open is 1 in 50 Seed set to 337 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[1] 1776190337.927110 mapwrite 0x38db60 thru 0x38dc15 (0xb6 bytes) 000003[0] 1776190338.088653 write 0x3f2f2d thru 0x3f390f (0x9e3 bytes) truncating to largest ever: 0x7d0a67 000006[1] 1776190338.204792 trunc from 0x3f3910 to 0x7d0a66 (0x3dd157 bytes) 000007[1] 1776190338.261190 mapwrite 0x3347c7 thru 0x33676b (0x1fa5 bytes) 000009[0] 1776190338.342354 write 0x43f9b9 thru 0x4404f6 (0xb3e bytes) 000011[0] 1776190338.405794 write 0x8b3783 thru 0x8b56e3 (0x1f61 bytes) 000012[0] 1776190338.426008 read 0x360bc9 thru 0x3617df (0xc17 bytes) 000013[1] 1776190338.454994 mapread 0x2bae81 thru 0x2bcd2a (0x1eaa bytes) 000014[0] 1776190338.545759 trunc from 0x8b56e4 to 0x3311f2 (0x5844f1 bytes) 000016[1] 1776190338.598645 mapread 0x1cb070 thru 0x1cca5c (0x19ed bytes) 000020[1] 1776190338.750226 write 0x38faa9 thru 0x3900e5 (0x63d bytes) 000021[1] 1776190338.791463 mapread 0x166f23 thru 0x1684b0 (0x158e bytes) 000022[0] 1776190338.850680 mapwrite 0x42c020 thru 0x42d120 (0x1101 bytes) 000023[0] 1776190339.035898 trunc from 0x42d121 to 0x0a4fcb (0x388155 bytes) 000024[1] 1776190339.085336 mapread 0x0a2b38 thru 0x0a42c8 (0x1791 bytes) 000026[0] 1776190339.166263 read 0x050b42 thru 0x0527c5 (0x1c84 bytes) 000027[0] 1776190339.204848 mapread 0x0443e5 thru 0x044fe0 (0xbfc bytes) 000029[0] 1776190339.290589 mapwrite 0x4800b3 thru 0x4802fb (0x249 bytes) 000032[0] 1776190339.443359 mapread 0x441363 thru 0x4416fa (0x398 bytes) 000036[0] 1776190339.496195 mapwrite 0x1bc0a9 thru 0x1bc24c (0x1a4 bytes) 000040[1] 1776190339.641985 read 0x250c6d thru 0x2518ae (0xc42 bytes) 000042[0] 1776190339.673189 trunc from 0x4802fc to 0x5dbfaa (0x15bcaf bytes) 000043[0] 1776190339.732057 mapwrite 0x14dcab thru 0x14de2d (0x183 bytes) 000045[0] 1776190339.827602 trunc from 0x5dbfab to 0x354f30 (0x28707a bytes) 000046[1] 1776190339.875171 write 0x467006 thru 0x468f13 (0x1f0e bytes) 000047[1] 1776190339.912759 read 0x0186c5 thru 0x01a02b (0x1967 bytes) 000048[1] 1776190339.958902 read 0x1003da thru 0x102284 (0x1eab bytes) 000049[0] 1776190340.054709 trunc from 0x468f14 to 0x58c5ab (0x123698 bytes) 000051[0] 1776190340.144599 write 0x72fb1c thru 0x7300c8 (0x5ad bytes) 000055[1] 1776190340.229879 mapread 0x3340d5 thru 0x335c91 (0x1bbd bytes) truncating to largest ever: 0x85a1a6 000056[1] 1776190340.282805 trunc from 0x7300c9 to 0x85a1a5 (0x12a0dd bytes) 000058[0] 1776190340.361860 read 0x5910a5 thru 0x591a63 (0x9bf bytes) 000059[1] 1776190340.381966 read 0x274f27 thru 0x275027 (0x101 bytes) 000060[1] 1776190340.465544 trunc from 0x85a1a6 to 0x0a1cf9 (0x7b84ac bytes) 000064[1] 1776190340.576978 trunc from 0x0a1cfa to 0x51b8e8 (0x479bef bytes) 000066[0] 1776190340.599470 mapread 0x083739 thru 0x084b85 (0x144d bytes) 000067[0] 1776190340.668981 write 0x5ffe6a thru 0x601a8c (0x1c23 bytes) 000068[1] 1776190340.711720 read 0x37ee53 thru 0x37f74b (0x8f9 bytes) 000069[1] 1776190340.753474 mapread 0x568e2c thru 0x569424 (0x5f9 bytes) 000071[0] 1776190340.794774 mapwrite 0x4feb9e thru 0x4ff9de (0xe41 bytes) 000072[0] 1776190340.861153 mapread 0x139c43 thru 0x13adc2 (0x1180 bytes) 000076[0] 1776190340.919986 read 0x08c624 thru 0x08d824 (0x1201 bytes) 000077[0] 1776190341.027941 read 0x5ecae3 thru 0x5ee00d (0x152b bytes) 000080[1] 1776190341.070554 mapwrite 0x5e53cf thru 0x5e557f (0x1b1 bytes) 000081[1] 1776190341.164627 read 0x138e52 thru 0x139589 (0x738 bytes) 000085[1] 1776190341.281616 read 0x58b0c1 thru 0x58cffb (0x1f3b bytes) 000087[0] 1776190341.347349 write 0x98cd1e thru 0x98d5b2 (0x895 bytes) 000090[1] 1776190341.418488 mapread 0x2a0d71 thru 0x2a15e9 (0x879 bytes) 000091[1] 1776190341.489437 read 0x15bb4e thru 0x15d78f (0x1c42 bytes) 000092[0] 1776190341.604755 mapwrite 0x736962 thru 0x737f92 (0x1631 bytes) 000094[0] 1776190341.683425 mapwrite 0x6b1bb3 thru 0x6b26b7 (0xb05 bytes) 000103[1] 1776190341.882940 write 0x536057 thru 0x53737a (0x1324 bytes) 000104[1] 1776190341.927918 write 0x5c6dd9 thru 0x5c78ae (0xad6 bytes) 000108[0] 1776190342.015664 write 0x75ee5a thru 0x760746 (0x18ed bytes) 000110[1] 1776190342.059647 write 0x034c33 thru 0x035967 (0xd35 bytes) 000111[1] 1776190342.140326 trunc from 0x98d5b3 to 0x827ba9 (0x165a09 bytes) 000114[0] 1776190342.183901 mapread 0x669c13 thru 0x66bbda (0x1fc8 bytes) 000118[0] 1776190342.297596 read 0x281d01 thru 0x2828c5 (0xbc5 bytes) 000119[1] 1776190342.325562 read 0x6b33f7 thru 0x6b51c1 (0x1dcb bytes) 000122[1] 1776190342.376651 read 0x0c0144 thru 0x0c1fc8 (0x1e85 bytes) 000123[1] 1776190342.496862 write 0x605fa0 thru 0x606b0c (0xb6d bytes) 000125[1] 1776190342.559912 read 0x421580 thru 0x422645 (0x10c6 bytes) 000126[0] 1776190342.613242 mapwrite 0x1d0885 thru 0x1d2582 (0x1cfe bytes) 000127[1] 1776190342.796792 read 0x6e7620 thru 0x6e95d1 (0x1fb2 bytes) 000130[0] 1776190342.848297 write 0x20081d thru 0x200ce9 (0x4cd bytes) 000131[0] 1776190342.962415 mapread 0x29d57d thru 0x29f1a6 (0x1c2a bytes) 000132[0] 1776190343.011410 read 0x229c39 thru 0x22b024 (0x13ec bytes) 000134[0] 1776190343.086101 mapwrite 0x9a5c09 thru 0x9a77eb (0x1be3 bytes) 000135[0] 1776190343.204291 write 0x501f5e thru 0x502642 (0x6e5 bytes) 000136[1] 1776190343.231913 write 0x743a41 thru 0x7456fd (0x1cbd bytes) 000137[0] 1776190343.302838 trunc from 0x9a77ec to 0x00a19d (0x99d64e bytes) 000139[0] 1776190343.422625 write 0x190b31 thru 0x191042 (0x512 bytes) 000143[0] 1776190343.445179 read 0x068958 thru 0x069d59 (0x1402 bytes) 000144[0] 1776190343.505473 write 0x107c47 thru 0x108dc9 (0x1183 bytes) 000146[0] 1776190343.506717 read 0x04273f thru 0x043108 (0x9ca bytes) 000148[1] 1776190343.555888 write 0x29fb71 thru 0x2a1081 (0x1511 bytes) 000151[1] 1776190343.631460 mapread 0x19abee thru 0x19c50e (0x1921 bytes) 000153[1] 1776190343.685833 mapread 0x054118 thru 0x05552b (0x1414 bytes) 000154[1] 1776190343.696850 mapread 0x13342e thru 0x133c90 (0x863 bytes) 000155[0] 1776190343.724415 mapread 0x0be75e thru 0x0c01ef (0x1a92 bytes) 000156[1] 1776190343.813512 mapread 0x16c9d7 thru 0x16d9f2 (0x101c bytes) 000157[1] 1776190343.825886 read 0x0259ae thru 0x02737d (0x19d0 bytes) 000160[0] 1776190343.874571 write 0x3dd075 thru 0x3dd923 (0x8af bytes) 000162[0] 1776190343.915099 mapwrite 0x327334 thru 0x328989 (0x1656 bytes) 000163[1] 1776190344.008958 write 0x5cdf39 thru 0x5cf862 (0x192a bytes) 000164[0] 1776190344.036784 mapwrite 0x5d7b11 thru 0x5d901d (0x150d bytes) 000167[1] 1776190344.246198 mapread 0x3758e8 thru 0x375c14 (0x32d bytes) 000169[1] 1776190344.299667 mapread 0x055875 thru 0x056295 (0xa21 bytes) 000171[1] 1776190344.382906 mapread 0x090ecd thru 0x092239 (0x136d bytes) 000172[0] 1776190344.403358 mapread 0x3ebc1c thru 0x3ed8a5 (0x1c8a bytes) 000175[1] 1776190344.436140 read 0x29a789 thru 0x29c0be (0x1936 bytes) 000176[0] 1776190344.452732 mapread 0x39bcec thru 0x39cfa9 (0x12be bytes) 000178[0] 1776190344.465828 write 0x03857e thru 0x039600 (0x1083 bytes) 000179[0] 1776190344.499977 mapwrite 0x457f04 thru 0x459090 (0x118d bytes) 000180[0] 1776190344.623001 mapread 0x2bafa9 thru 0x2bc46e (0x14c6 bytes) truncating to largest ever: 0x86c0f0 000182[0] 1776190344.642439 trunc from 0x5d901e to 0x86c0ef (0x2930d2 bytes) 000185[0] 1776190344.696877 read 0x6d6c64 thru 0x6d7f53 (0x12f0 bytes) 000186[1] 1776190344.744838 mapread 0x40956d thru 0x40a445 (0xed9 bytes) 000187[0] 1776190344.794544 mapread 0x348d58 thru 0x349adf (0xd88 bytes) 000190[0] 1776190344.878398 mapread 0x34f6b4 thru 0x350b33 (0x1480 bytes) 000191[1] 1776190344.899558 mapwrite 0x5006f4 thru 0x5019a5 (0x12b2 bytes) 000195[1] 1776190345.039836 read 0x7ee488 thru 0x7ef70a (0x1283 bytes) 000196[1] 1776190345.040664 read 0x680956 thru 0x68116d (0x818 bytes) 000197[0] 1776190345.041062 read 0x09341e thru 0x093fee (0xbd1 bytes) 000199[0] 1776190345.058860 read 0x1430d9 thru 0x1441a2 (0x10ca bytes) 000201[1] 1776190345.187974 write 0x2b1c15 thru 0x2b286e (0xc5a bytes) 000202[1] 1776190345.266375 mapwrite 0x5f9d6c thru 0x5fa7c7 (0xa5c bytes) 000203[0] 1776190345.447567 mapread 0x7817fd thru 0x782443 (0xc47 bytes) 000204[0] 1776190345.527798 write 0x930723 thru 0x9323a0 (0x1c7e bytes) 000205[1] 1776190345.578197 mapwrite 0x4d7a6f thru 0x4d86fb (0xc8d bytes) 000208[1] 1776190345.683471 read 0x26a140 thru 0x26ac37 (0xaf8 bytes) 000209[1] 1776190345.743402 trunc from 0x9323a1 to 0x26b8c3 (0x6c6add bytes) 000213[1] 1776190345.845300 write 0x13eb92 thru 0x13f2b5 (0x724 bytes) 000215[1] 1776190345.876195 write 0x168722 thru 0x168d88 (0x667 bytes) 000216[1] 1776190345.899770 mapwrite 0x6dae87 thru 0x6dc795 (0x190f bytes) 000217[0] 1776190346.117124 trunc from 0x6dc796 to 0x4f6bf2 (0x1e5ba3 bytes) 000220[0] 1776190346.284509 write 0x5f67cf thru 0x5f7555 (0xd87 bytes) 000221[0] 1776190346.313107 trunc from 0x5f7556 to 0x0cc299 (0x52b2bc bytes) 000228[0] 1776190346.517098 read 0x028497 thru 0x02a443 (0x1fad bytes) 000229[1] 1776190346.542004 read 0x0c6793 thru 0x0c6e42 (0x6b0 bytes) 000233[1] 1776190346.611989 write 0x538af2 thru 0x53a658 (0x1b67 bytes) 000236[0] 1776190346.649837 mapread 0x19c936 thru 0x19d805 (0xed0 bytes) 000237[1] 1776190346.765293 mapwrite 0x795610 thru 0x797541 (0x1f32 bytes) 000240[0] 1776190347.092306 mapwrite 0x9724a6 thru 0x972e35 (0x990 bytes) 000243[0] 1776190347.318766 write 0x0d8c4c thru 0x0dab7c (0x1f31 bytes) 000244[0] 1776190347.368156 read 0x636fb9 thru 0x637982 (0x9ca bytes) 000245[0] 1776190347.418661 read 0x70be88 thru 0x70c963 (0xadc bytes) 000247[1] 1776190347.443852 mapwrite 0x739658 thru 0x739f68 (0x911 bytes) 000248[1] 1776190347.549959 write 0x6f0653 thru 0x6f1614 (0xfc2 bytes) 000249[1] 1776190347.604127 trunc from 0x972e36 to 0x2d2278 (0x6a0bbd bytes) 000250[1] 1776190347.626348 write 0x68aa90 thru 0x68b3a2 (0x913 bytes) 000251[0] 1776190347.660568 read 0x372b01 thru 0x373dc6 (0x12c6 bytes) 000252[0] 1776190347.706695 mapread 0x2b5cea thru 0x2b6673 (0x98a bytes) 000256[0] 1776190347.791753 mapread 0x382076 thru 0x384046 (0x1fd1 bytes) 000257[0] 1776190347.792788 read 0x6865d8 thru 0x687be1 (0x160a bytes) 000260[1] 1776190347.795650 write 0x65d0df thru 0x65eb05 (0x1a27 bytes) 000263[1] 1776190347.856294 read 0x622cad thru 0x623227 (0x57b bytes) 000264[0] 1776190347.887323 mapread 0x2e0550 thru 0x2e13de (0xe8f bytes) 000265[1] 1776190347.926135 trunc from 0x68b3a3 to 0x05c5e3 (0x62edbf bytes) 000266[1] 1776190347.978427 mapread 0x031343 thru 0x0323ac (0x106a bytes) 000267[1] 1776190347.994397 trunc from 0x05c5e4 to 0x558fec (0x4fca09 bytes) 000268[1] 1776190348.029928 mapwrite 0x74c5a8 thru 0x74d4c7 (0xf20 bytes) 000270[1] 1776190348.171774 mapread 0x0f4572 thru 0x0f5524 (0xfb3 bytes) 000271[1] 1776190348.184730 mapwrite 0x672680 thru 0x6742eb (0x1c6c bytes) 000273[0] 1776190348.250952 mapread 0x54aeaf thru 0x54bea2 (0xff4 bytes) 000275[1] 1776190348.309954 read 0x143473 thru 0x144362 (0xef0 bytes) 000277[0] 1776190348.422089 mapread 0x5dc974 thru 0x5dde13 (0x14a0 bytes) 000278[0] 1776190348.444856 write 0x8319de thru 0x8333c2 (0x19e5 bytes) 000279[1] 1776190348.491815 mapwrite 0x3e0f15 thru 0x3e24a0 (0x158c bytes) 000281[0] 1776190348.712180 read 0x53ceb9 thru 0x53dbb1 (0xcf9 bytes) 000282[0] 1776190348.831753 trunc from 0x8333c3 to 0x539986 (0x2f9a3c bytes) 000283[0] 1776190348.881914 mapread 0x3f1128 thru 0x3f163e (0x517 bytes) 000284[0] 1776190348.931130 write 0x674aeb thru 0x6767ea (0x1d00 bytes) 000286[0] 1776190348.957010 mapwrite 0x8d0a93 thru 0x8d179f (0xd0d bytes) 000289[0] 1776190349.075883 write 0x263b3a thru 0x265a83 (0x1f4a bytes) 000290[1] 1776190349.118484 read 0x314b8c thru 0x31659e (0x1a13 bytes) 000291[0] 1776190349.179321 mapread 0x739e36 thru 0x73b6db (0x18a6 bytes) 000292[0] 1776190349.242830 read 0x1a954d thru 0x1aae84 (0x1938 bytes) 000293[0] 1776190349.352702 mapread 0x53d879 thru 0x53f2b0 (0x1a38 bytes) 000294[0] 1776190349.353641 trunc from 0x8d17a0 to 0x08cac2 (0x844cdd bytes) 000296[0] 1776190349.475643 write 0x4a6d6f thru 0x4a896d (0x1bff bytes) 000298[1] 1776190349.527273 write 0x2cae01 thru 0x2cb859 (0xa59 bytes) 000301[1] 1776190349.641662 mapwrite 0x3ba050 thru 0x3ba97d (0x92e bytes) 000302[1] 1776190349.730656 read 0x33326b thru 0x333f5c (0xcf2 bytes) 000303[1] 1776190349.750972 mapwrite 0x4ab51b thru 0x4ac0f0 (0xbd6 bytes) 000304[1] 1776190349.852885 read 0x453eff thru 0x454e44 (0xf46 bytes) 000307[0] 1776190349.930165 trunc from 0x4ac0f1 to 0x3442a3 (0x167e4d bytes) 000308[0] 1776190350.003220 read 0x230507 thru 0x231393 (0xe8d bytes) 000309[0] 1776190350.039514 mapread 0x1b76ea thru 0x1b7e18 (0x72f bytes) 000310[1] 1776190350.094605 write 0x384782 thru 0x385e42 (0x16c1 bytes) 000311[1] 1776190350.125741 trunc from 0x385e43 to 0x3a4978 (0x1eb36 bytes) 000312[1] 1776190350.164870 trunc from 0x3a4979 to 0x311662 (0x93316 bytes) 000313[0] 1776190350.185657 mapread 0x29d81e thru 0x29dc96 (0x479 bytes) 000314[0] 1776190350.242844 write 0x6b2556 thru 0x6b38f2 (0x139d bytes) 000315[0] 1776190350.271577 read 0x61faff thru 0x61fe56 (0x358 bytes) 000316[1] 1776190350.332707 mapwrite 0x3bc0a0 thru 0x3bd380 (0x12e1 bytes) 000318[0] 1776190350.444379 trunc from 0x6b38f3 to 0x603a3d (0xafeb5 bytes) 000319[0] 1776190350.486255 mapread 0x110ac8 thru 0x1117ba (0xcf3 bytes) 000321[1] 1776190350.506767 mapwrite 0x8587c6 thru 0x858b8e (0x3c9 bytes) 000324[1] 1776190350.815193 read 0x073015 thru 0x073edb (0xec7 bytes) 000325[1] 1776190350.918162 mapwrite 0x708add thru 0x70a0d4 (0x15f8 bytes) 000327[0] 1776190351.002675 mapread 0x30507a thru 0x305978 (0x8ff bytes) 000329[0] 1776190351.075694 trunc from 0x858b8f to 0x44f823 (0x40936b bytes) 000330[1] 1776190351.166012 trunc from 0x44f824 to 0x3b3f94 (0x9b88f bytes) 000332[0] 1776190351.238815 trunc from 0x3b3f95 to 0x344f71 (0x6f023 bytes) 000334[0] 1776190351.342594 read 0x302674 thru 0x30340c (0xd99 bytes) 000336[0] 1776190351.385380 write 0x5d12ab thru 0x5d1751 (0x4a7 bytes) 000337[0] 1776190351.388776 trunc from 0x5d1752 to 0x2b2051 (0x31f700 bytes) 000339[0] 1776190351.416696 mapwrite 0x2c4664 thru 0x2c60ad (0x1a4a bytes) 000340[0] 1776190351.541888 mapwrite 0x9538b3 thru 0x9539ca (0x118 bytes) 000341[0] 1776190351.821873 write 0x118da9 thru 0x11a547 (0x179f bytes) 000345[1] 1776190351.926859 mapwrite 0x860807 thru 0x861e39 (0x1633 bytes) 000346[1] 1776190352.049709 read 0x0b8256 thru 0x0b8915 (0x6c0 bytes) 000347[1] 1776190352.095639 mapwrite 0x2a53e5 thru 0x2a7111 (0x1d2d bytes) 000349[1] 1776190352.220941 read 0x70cc8a thru 0x70deb5 (0x122c bytes) 000352[0] 1776190352.332627 trunc from 0x9539cb to 0x24feb1 (0x703b19 bytes) 000354[0] 1776190352.420920 read 0x05d650 thru 0x05f53c (0x1eed bytes) 000357[0] 1776190352.537092 trunc from 0x24feb2 to 0x466bf4 (0x216d43 bytes) 000358[1] 1776190352.586984 trunc from 0x466bf5 to 0x4d9dae (0x731ba bytes) 000360[1] 1776190352.656441 write 0x401289 thru 0x40225b (0xfd3 bytes) 000362[0] 1776190352.693759 mapread 0x0d1089 thru 0x0d24b0 (0x1428 bytes) 000363[0] 1776190352.766622 mapwrite 0x1ced65 thru 0x1cf7d2 (0xa6e bytes) 000365[1] 1776190352.982493 read 0x383b25 thru 0x384a33 (0xf0f bytes) 000366[1] 1776190353.017738 mapread 0x108508 thru 0x1099e3 (0x14dc bytes) 000369[0] 1776190353.041734 write 0x350d0d thru 0x352cc8 (0x1fbc bytes) 000370[0] 1776190353.083431 mapread 0x4d4456 thru 0x4d559b (0x1146 bytes) 000371[0] 1776190353.086189 mapread 0x086253 thru 0x086a33 (0x7e1 bytes) 000372[0] 1776190353.122306 trunc from 0x4d9daf to 0x2d9d10 (0x20009e bytes) 000373[1] 1776190353.196090 read 0x16db95 thru 0x16e117 (0x583 bytes) 000377[1] 1776190353.268467 mapwrite 0x87f8de thru 0x8803b1 (0xad4 bytes) 000379[0] 1776190353.571745 trunc from 0x8803b2 to 0x80aa74 (0x7593d bytes) 000380[1] 1776190353.713313 trunc from 0x80aa75 to 0x6e5894 (0x1251e0 bytes) 000382[0] 1776190353.835889 write 0x7f9c01 thru 0x7f9cda (0xda bytes) 000384[1] 1776190353.892558 write 0x109d24 thru 0x109e9b (0x178 bytes) 000386[0] 1776190353.926886 write 0x5836f3 thru 0x5846c0 (0xfce bytes) 000387[1] 1776190353.969485 read 0x593320 thru 0x595060 (0x1d41 bytes) 000388[1] 1776190354.020369 mapwrite 0x99b993 thru 0x99d4d5 (0x1b43 bytes) truncating to largest ever: 0x9d3e83 000389[0] 1776190354.149737 trunc from 0x99d4d6 to 0x9d3e82 (0x369ad bytes) 000394[1] 1776190354.259552 write 0x99c07a thru 0x99dfb4 (0x1f3b bytes) 000396[1] 1776190354.346860 trunc from 0x9d3e83 to 0x67f056 (0x354e2c bytes) 000398[0] 1776190354.404480 trunc from 0x67f057 to 0x396efa (0x2e815c bytes) 000401[0] 1776190354.553559 mapread 0x12f1c1 thru 0x13042f (0x126f bytes) 000402[1] 1776190354.588707 read 0x18cc8c thru 0x18cec1 (0x236 bytes) 000403[1] 1776190354.625858 mapwrite 0x6054f8 thru 0x606774 (0x127d bytes) 000404[0] 1776190354.964395 read 0x0f7f46 thru 0x0f8d53 (0xe0e bytes) 000405[1] 1776190355.001792 read 0x2737d2 thru 0x273cf2 (0x521 bytes) 000406[1] 1776190355.052530 trunc from 0x606775 to 0x59077c (0x75ff8 bytes) 000407[0] 1776190355.095955 trunc from 0x59077d to 0x3afeb4 (0x1e08c8 bytes) 000408[0] 1776190355.154785 mapread 0x0dbd03 thru 0x0dc760 (0xa5e bytes) 000411[0] 1776190355.281975 read 0x038223 thru 0x038847 (0x625 bytes) 000412[1] 1776190355.341837 read 0x236028 thru 0x236629 (0x602 bytes) 000415[1] 1776190355.420119 read 0x158b05 thru 0x15a826 (0x1d22 bytes) 000418[0] 1776190355.508700 write 0x6f5329 thru 0x6f6223 (0xefb bytes) 000419[0] 1776190355.561621 trunc from 0x6f6224 to 0x7e7691 (0xf146e bytes) 000420[1] 1776190355.622988 read 0x709a34 thru 0x70ac68 (0x1235 bytes) 000421[0] 1776190355.699331 mapread 0x232009 thru 0x23335f (0x1357 bytes) 000423[0] 1776190355.827519 read 0x361c2a thru 0x361c3c (0x13 bytes) 000424[0] 1776190355.848473 trunc from 0x7e7692 to 0x5da58f (0x20d102 bytes) 000426[1] 1776190355.945691 read 0x0d8ccb thru 0x0d93b3 (0x6e9 bytes) 000427[0] 1776190356.033523 mapread 0x50be1f thru 0x50d1ce (0x13b0 bytes) 000428[1] 1776190356.097253 write 0x4ee3f8 thru 0x4eec9e (0x8a7 bytes) 000429[0] 1776190356.141705 mapwrite 0x74ce95 thru 0x74d005 (0x171 bytes) 000431[0] 1776190356.331959 trunc from 0x74d006 to 0x3ff65e (0x34d9a7 bytes) 000432[1] 1776190356.374851 mapread 0x19aee3 thru 0x19b371 (0x48f bytes) 000433[0] 1776190356.445410 mapread 0x0dc901 thru 0x0dcbc1 (0x2c1 bytes) 000434[1] 1776190356.476914 write 0x82c3f3 thru 0x82cc18 (0x826 bytes) 000437[0] 1776190356.523966 write 0x249e33 thru 0x24b41c (0x15ea bytes) 000438[1] 1776190356.566970 read 0x0a0303 thru 0x0a09c9 (0x6c7 bytes) 000439[0] 1776190356.725785 trunc from 0x82cc19 to 0x6a7fe5 (0x184c33 bytes) 000440[1] 1776190356.851806 trunc from 0x6a7fe6 to 0x1c01c4 (0x4e7e21 bytes) 000441[1] 1776190356.931598 write 0x0880e6 thru 0x089b6b (0x1a86 bytes) 000443[1] 1776190356.981282 read 0x0ec418 thru 0x0ed9d4 (0x15bd bytes) 000445[0] 1776190357.042740 mapwrite 0x8f95c1 thru 0x8f9f8c (0x9cc bytes) 000447[1] 1776190357.280573 mapread 0x08f27e thru 0x08f2d4 (0x57 bytes) 000448[0] 1776190357.333783 write 0x918a14 thru 0x91a80b (0x1df8 bytes) 000449[0] 1776190357.340611 read 0x25f549 thru 0x2606e5 (0x119d bytes) 000451[0] 1776190357.450330 write 0x616ddf thru 0x618d6a (0x1f8c bytes) 000453[1] 1776190357.516705 mapwrite 0x54c8a9 thru 0x54e1af (0x1907 bytes) 000455[1] 1776190357.680795 mapwrite 0x2a8bf8 thru 0x2aa112 (0x151b bytes) 000457[0] 1776190357.845793 write 0x160035 thru 0x161d29 (0x1cf5 bytes) 000458[0] 1776190357.893823 mapwrite 0x27ea29 thru 0x27ff07 (0x14df bytes) 000459[1] 1776190358.030796 write 0x9ad54f thru 0x9aedcf (0x1881 bytes) 000460[1] 1776190358.075496 trunc from 0x9aedd0 to 0x7e6608 (0x1c87c7 bytes) 000461[0] 1776190358.123984 mapread 0x130e6f thru 0x131c80 (0xe12 bytes) 000463[1] 1776190358.195620 mapread 0x0086e8 thru 0x0087ce (0xe7 bytes) 000466[1] 1776190358.207848 read 0x5e6d16 thru 0x5e7f1c (0x1207 bytes) 000467[1] 1776190358.285724 write 0x4fd7cd thru 0x4fe1d6 (0xa0a bytes) 000470[1] 1776190358.313385 mapread 0x729dad thru 0x72a51b (0x76f bytes) 000475[0] 1776190358.317143 mapwrite 0x6aa554 thru 0x6ab667 (0x1114 bytes) 000477[1] 1776190358.439344 mapwrite 0x84ad49 thru 0x84bdf2 (0x10aa bytes) 000480[1] 1776190358.590961 mapread 0x538b8c thru 0x53a8b2 (0x1d27 bytes) 000482[1] 1776190358.652639 mapwrite 0x13792d thru 0x137fd7 (0x6ab bytes) 000483[0] 1776190358.720692 write 0x848dd7 thru 0x849439 (0x663 bytes) 000484[1] 1776190358.787531 trunc from 0x84bdf3 to 0x62afa8 (0x220e4a bytes) 000487[0] 1776190358.922905 mapwrite 0x389224 thru 0x389971 (0x74e bytes) 000488[1] 1776190359.081453 trunc from 0x62afa9 to 0x4241a6 (0x206e02 bytes) 000490[0] 1776190359.111409 mapread 0x22c7c7 thru 0x22d6da (0xf14 bytes) 000491[1] 1776190359.151561 read 0x396e82 thru 0x397ad2 (0xc51 bytes) 000492[1] 1776190359.176625 write 0x986beb thru 0x986ed0 (0x2e6 bytes) 000493[1] 1776190359.202921 read 0x74c160 thru 0x74dcb9 (0x1b5a bytes) 000498[0] 1776190359.313937 mapread 0x4209ac thru 0x422059 (0x16ae bytes) 000499[1] 1776190359.362669 mapwrite 0x45cf53 thru 0x45d177 (0x225 bytes) All operations completed A-OK! PASS 16b (28s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 14:12:43 (1776190363) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (3s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 14:12:46 (1776190366) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.233692 s, 9.0 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.168657 s, 12.4 MB/s ldlm.namespaces.MGC192.168.203.114@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff964048299800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff964060cdb800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff964060cdb800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff964060cdb800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.70807 s, 12.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.49502 s, 42.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.75327 s, 59.8 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.27001 s, 24.6 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.73512 s, 18.3 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.38878 s, 43.9 MB/s PASS 16d (35s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 14:13:21 (1776190401) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.322573 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126263 s, 8.3 MB/s PASS 16e (5s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 14:13:26 (1776190406) TIMEOUT PASS 16f (28s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 14:13:55 (1776190435) TIMEOUT PASS 16g (28s) == sanityn test 16i: read after truncate file ============ 14:14:23 (1776190463) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (7s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 14:14:30 (1776190470) fail_loc=0x8000030a PASS 17 (10s) == sanityn test 18: mmap sanity check =========================================================================================== 14:14:40 (1776190480) mmap test1: basic mmap operation (PASS, 0.13406s) mmap test2: MAP_PRIVATE not write back (PASS, 0.064091s) mmap test3: concurrent mmap ops on two nodes (PASS, 8.4338s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.4293s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.11074s) mmap test6: check mmap write/read content on two nodes (PASS, 0.16796s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.1007s) PASS 18 (19s) == sanityn test 19: test concurrent uncached read races ========================================================================= 14:14:59 (1776190499) oleg314-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg314-client: oleg314-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (3s) == sanityn test 20: test extra readahead page left in cache ============================================================== 14:15:02 (1776190502) PASS 20 (6s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 14:15:08 (1776190508) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (6s) == sanityn test 23: others should see updated atime while another read============================================================== 14:15:14 (1776190514) now is 1776190516 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 new atime is 1776190577 PASS 23 (69s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 14:16:23 (1776190583) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 374 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 374 115.0K 1% /mnt/lustre[OST:1] filesystem_summary: 230.2K 368 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 374 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 374 115.0K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 368 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 7.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 4.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 11.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118006 374 117632 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118102 374 117728 1% /mnt/lustre2[OST:1] filesystem_summary: 235728 368 235360 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 374 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 374 115.0K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 368 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118006 374 117632 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118102 374 117728 1% /mnt/lustre[OST:1] filesystem_summary: 235728 368 235360 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118006 374 117632 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118102 374 117728 1% /mnt/lustre2[OST:1] filesystem_summary: 235728 368 235360 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 PASS 24a (7s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 14:16:31 (1776190591) PASS 24b (6s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 14:16:38 (1776190598) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (7s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 14:16:45 (1776190605) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (3s) == sanityn test 26a: allow mtime to get older ============ 14:16:48 (1776190608) utime: good mknod times 1776190608 <= 1776190609 <= 1776190609 for /mnt/lustre/f26a utime: good mknod times 1776190608 <= 1776190609 <= 1776190609 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (5s) == sanityn test 26b: sync mtime between ost and mds ====== 14:16:53 (1776190613) PASS 26b (7s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 14:17:03 (1776190623) 0 PASS 30 (7s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 14:17:10 (1776190630) fail_loc=0x314 PASS 31a (6s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 14:17:17 (1776190637) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg314-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.203.114@tcp:/lustre 7542784 12288 7524352 1% /mnt/lustre2 PASS 31b (21s) == sanityn test 31r: open-rename(replace) race =========== 14:17:37 (1776190657) -rw-r--r-- 1 root root 0 Apr 14 14:17 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (9s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 14:17:47 (1776190667) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (3s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 14:17:50 (1776190670) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (2s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 14:17:52 (1776190672) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (3s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 14:17:55 (1776190675) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (2s) == sanityn test 34: no lock timeout under IO ============= 14:17:57 (1776190677) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg314-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff964048299800.ost_server_uuid,osc.lustre-OST0000-osc-ffff964060cdb800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff964048299800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff964060cdb800.ost_server_uuid in FULL state after 0 sec oleg314-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff964048299800.ost_server_uuid,osc.lustre-OST0001-osc-ffff964060cdb800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff964048299800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff964060cdb800.ost_server_uuid in IDLE state after 0 sec oleg314-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff964048299800.ost_server_uuid,osc.lustre-OST0000-osc-ffff964060cdb800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff964048299800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff964060cdb800.ost_server_uuid in FULL state after 0 sec oleg314-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff964048299800.ost_server_uuid,osc.lustre-OST0001-osc-ffff964060cdb800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff964048299800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff964060cdb800.ost_server_uuid in IDLE state after 0 sec oleg314-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff964048299800.ost_server_uuid,osc.lustre-OST0000-osc-ffff964060cdb800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff964048299800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff964060cdb800.ost_server_uuid in FULL state after 0 sec oleg314-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff964048299800.ost_server_uuid,osc.lustre-OST0001-osc-ffff964060cdb800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff964048299800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff964060cdb800.ost_server_uuid in IDLE state after 0 sec PASS 34 (124s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 14:20:01 (1776190801) fail_loc=0x80000317 Race attempt 0 Wait for 44687 44732 for 60 sec... - open/close 692 (time 1776190816.59 total 10.01 last 69.13) - open/close 1403 (time 1776190826.59 total 20.02 last 71.06) - open/close 2111 (time 1776190836.60 total 30.03 last 70.72) - open/close 2934 (time 1776190846.61 total 40.03 last 82.26) - open/close 3708 (time 1776190856.62 total 50.04 last 77.33) total: 4000 open/close in 53.88 seconds: 74.24 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.203.114@tcp:/lustre 7.2G 13M 7.2G 1% /mnt/lustre 192.168.203.114@tcp:/lustre 7.2G 13M 7.2G 1% /mnt/lustre2 PASS 35 (71s) == sanityn test 36: handle ESTALE/open-unlink correctly == 14:21:12 (1776190872) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg314-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.4248 s, 36.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.26566 s, 41.4 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.991938 s, 52.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.956484 s, 54.8 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7525376) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 2.07261 s, 25.3 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS Waiting for MDT destroys to complete *** cycle(4) *** before(7523328) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.39197 s, 37.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7525376) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.13727 s, 46.1 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7523328) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.875228 s, 59.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7525376) after_dd(7474176) after(7525376) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.868589 s, 60.4 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7525376) after_dd(7470080) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 1.16197 s, 45.1 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7523328) after_dd(7470080) after(7521280) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.813151 s, 64.5 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7521280) after_dd(7474176) after(7525376) PASS 36 (290s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 14:26:03 (1776191163) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 - create 1535 (time 1776191176.17 total 10.01 last 153.36) - create 3144 (time 1776191186.18 total 20.02 last 160.79) - create 4390 (time 1776191196.18 total 30.02 last 124.56) - create 5541 (time 1776191206.18 total 40.02 last 115.09) - create 6710 (time 1776191216.18 total 50.02 last 116.89) - create 8091 (time 1776191226.20 total 60.04 last 137.88) - create 9488 (time 1776191236.21 total 70.05 last 139.54) total: 10000 create in 73.75 seconds: 135.60 ops/second PASS 37 (83s) == sanityn test 39a: test from 11063 ============================================================================================ 14:27:26 (1776191246) repeat after cancel_lru_locks PASS 39a (8s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 14:27:35 (1776191255) repeat after cancel_lru_locks PASS 39b (9s) == sanityn test 39c: check truncate mtime update ================================================================================ 14:27:45 (1776191265) repeat after cancel_lru_locks PASS 39c (8s) == sanityn test 39d: sync write should update mtime ====== 14:27:53 (1776191273) fail_loc=0x411 fail_loc=0 PASS 39d (7s) == sanityn test 40a: pdirops: create vs others ======================================================================== 14:28:00 (1776191280) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (18s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 14:28:18 (1776191298) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (15s) == sanityn test 40c: pdirops: link and others ======================================================================== 14:28:34 (1776191314) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (16s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 14:28:50 (1776191330) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (13s) == sanityn test 40e: pdirops: rename and others ======================================================================== 14:29:04 (1776191344) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (14s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 14:29:18 (1776191358) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (12s) == sanityn test 41b: pdirops: create vs create ======================================================================== 14:29:30 (1776191370) open: File exists Conflict PASS 41b (12s) == sanityn test 41c: pdirops: create vs link ======================================================================== 14:29:42 (1776191382) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (11s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 14:29:54 (1776191394) Conflict PASS 41d (11s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 14:30:06 (1776191406) Conflict PASS 41e (11s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 14:30:17 (1776191417) Conflict PASS 41f (12s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 14:30:30 (1776191430) Conflict PASS 41g (12s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 14:30:42 (1776191442) Conflict PASS 41h (14s) == sanityn test 41i: reint_open: create vs create ======== 14:30:56 (1776191456) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (1070s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 14:48:46 (1776192526) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (15s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 14:49:01 (1776192541) open: File exists Conflict PASS 42b (16s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 14:49:18 (1776192558) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (16s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 14:49:34 (1776192574) Conflict PASS 42d (14s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 14:49:48 (1776192588) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (13s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 14:50:01 (1776192601) Conflict PASS 42f (22s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 14:50:23 (1776192623) Conflict PASS 42g (13s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 14:50:37 (1776192637) Conflict PASS 42h (11s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 14:50:49 (1776192649) PASS 43a (63s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 14:51:53 (1776192713) Conflict PASS 43b (15s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 14:52:07 (1776192727) Conflict PASS 43c (16s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 14:52:23 (1776192743) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (14s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 14:52:37 (1776192757) Conflict PASS 43e (13s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 14:52:50 (1776192770) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (13s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 14:53:03 (1776192783) stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (12s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 14:53:15 (1776192795) Conflict PASS 43h (13s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 14:53:28 (1776192808) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (3s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 14:53:31 (1776192811) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (153s) == sanityn test 43k: unlink vs create ==================== 14:56:05 (1776192965) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (1262s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 15:17:06 (1776194226) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Tue Apr 14 15:17:14 EDT 2026 PASS 44a (12s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 15:17:18 (1776194238) open: File exists Conflict PASS 44b (12s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 15:17:30 (1776194250) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (12s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 15:17:42 (1776194262) Conflict PASS 44d (12s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 15:17:54 (1776194274) Conflict PASS 44e (12s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 15:18:07 (1776194287) Conflict PASS 44f (14s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 15:18:21 (1776194301) Conflict PASS 44g (15s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 15:18:36 (1776194316) Conflict PASS 44h (15s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 15:18:51 (1776194331) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (3s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 15:18:54 (1776194334) PASS 45a (119s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 15:20:53 (1776194453) Conflict PASS 45b (13s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 15:21:06 (1776194466) Conflict PASS 45c (14s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 15:21:21 (1776194481) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (13s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 15:21:33 (1776194493) Conflict PASS 45e (13s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 15:21:46 (1776194506) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (13s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 15:21:59 (1776194519) stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (13s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 15:22:12 (1776194532) Conflict PASS 45h (13s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 15:22:25 (1776194545) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (3s) == sanityn test 45j: read vs rename ====================== 15:22:28 (1776194548) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (1248s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 15:43:16 (1776195796) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (15s) == sanityn test 46b: pdirops: link vs create ======================================================================== 15:43:31 (1776195811) open: File exists Conflict PASS 46b (14s) == sanityn test 46c: pdirops: link vs link ======================================================================== 15:43:45 (1776195825) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (12s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 15:43:58 (1776195838) Conflict PASS 46d (13s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 15:44:11 (1776195851) Conflict PASS 46e (14s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 15:44:25 (1776195865) Conflict PASS 46f (12s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 15:44:38 (1776195878) Conflict PASS 46g (11s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 15:44:50 (1776195890) Conflict PASS 46h (14s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 15:45:03 (1776195903) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (3s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 15:45:07 (1776195907) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (2s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 15:45:10 (1776195910) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (3s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 15:45:13 (1776195913) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (3s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 15:45:16 (1776195916) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (3s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 15:45:19 (1776195919) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (4s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 15:45:23 (1776195923) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (4s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 15:45:28 (1776195928) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (5s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 15:45:32 (1776195932) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.139792 s, 73.3 kB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (12s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 15:45:44 (1776195944) 0+1 records in 0+1 records out 158 bytes copied, 0.0473399 s, 3.3 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (242482) - No such process PASS 51a (9s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 15:45:53 (1776195953) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0141751 s, 72.2 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0745887 s, 14.1 MB/s 3145728 PASS 51b (19s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 15:46:12 (1776195972) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0284804 s, 36.0 kB/s PASS 51c (11s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 15:46:24 (1776195984) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0640921 s, 16.4 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (9s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 15:46:33 (1776195993) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2432 0x980 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2432 0x980 0 read lease(1) has applied. PASS 51e (10s) == sanityn test 54: rename locking ======================= 15:46:43 (1776196003) llite.lustre-ffff964048299800.xattr_cache=1 llite.lustre-ffff964060cdb800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504962 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-04-14 15:46:45.000000000 -0400 Modify: 2026-04-14 15:46:45.000000000 -0400 Change: 2026-04-14 15:46:45.000000000 -0400 Birth: 2026-04-14 15:46:46.000000000 -0400 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504965 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-04-14 15:46:52.000000000 -0400 Modify: 2026-04-14 15:46:53.000000000 -0400 Change: 2026-04-14 15:46:53.000000000 -0400 Birth: 2026-04-14 15:46:53.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-04-14 15:46:59.000000000 -0400 Modify: 2026-04-14 15:46:59.000000000 -0400 Change: 2026-04-14 15:46:59.000000000 -0400 Birth: 2026-04-14 15:46:59.000000000 -0400 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504967 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-04-14 15:46:59.000000000 -0400 Modify: 2026-04-14 15:47:00.000000000 -0400 Change: 2026-04-14 15:47:00.000000000 -0400 Birth: 2026-04-14 15:46:59.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504970 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-04-14 15:47:05.000000000 -0400 Modify: 2026-04-14 15:47:06.000000000 -0400 Change: 2026-04-14 15:47:06.000000000 -0400 Birth: 2026-04-14 15:47:06.000000000 -0400 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-04-14 15:47:05.000000000 -0400 Modify: 2026-04-14 15:47:06.000000000 -0400 Change: 2026-04-14 15:47:06.000000000 -0400 Birth: 2026-04-14 15:47:06.000000000 -0400 PASS 54 (33s) == sanityn test 55a: rename vs unlink target dir ========= 15:47:17 (1776196037) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (12s) == sanityn test 55b: rename vs unlink source dir ========= 15:47:29 (1776196049) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (13s) == sanityn test 55c: rename vs unlink orphan target dir == 15:47:42 (1776196062) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (18s) == sanityn test 55d: rename file vs link ================= 15:48:00 (1776196080) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (15s) == sanityn test 60: Verify data_version behaviour ======== 15:48:15 (1776196095) PASS 60 (6s) == sanityn test 70a: cd directory && rm directory ======== 15:48:21 (1776196101) PASS 70a (6s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 15:48:27 (1776196107) PASS 70b (8s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 15:48:35 (1776196115) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0338802 s, 1.9 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (3s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 15:48:39 (1776196119) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.010685 s, 6.1 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (3s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 15:48:42 (1776196122) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (3s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 15:48:46 (1776196126) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0203581 s, 3.2 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (3s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 15:48:49 (1776196129) llite.lustre-ffff964048299800.xattr_cache=1 llite.lustre-ffff964060cdb800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (6s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 15:48:55 (1776196135) llite.lustre-ffff964048299800.xattr_cache=1 llite.lustre-ffff964060cdb800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (8s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 15:49:03 (1776196143) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0198691 s, 51.5 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0339633 s, 30.2 kB/s 252532: get lock2 252533: get lock1 252532: done 252532: get lock1 252533: done 252533: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 252533: cannot get lock2: Resource deadlock avoided 252533: exit rc=0 252532: done 252532: exit rc=0 PASS 74 (10s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 15:49:13 (1776196153) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0737209 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028994 s, 36.2 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 253107 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (12s) == sanityn test 76: Verify MDT open_files listing ======== 15:49:25 (1776196165) opening files: ........ closing files: ........ PASS 76 (72s) == sanityn test 77a: check FIFO NRS policy =============== 15:50:37 (1776196237) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.364829 s, 46.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0654648 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0761587 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160373 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.212989 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.21154 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201317 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.236985 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217835 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273648 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.261603 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.314723 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.331955 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.328498 s, 3.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.348807 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.321994 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.401862 s, 2.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0508735 s, 20.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262299 s, 40.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315558 s, 33.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278464 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.037103 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0619679 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0774756 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0370447 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0434066 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0438297 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0981422 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0470047 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0855599 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0770225 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0708087 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0660552 s, 15.9 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (8s) == sanityn test 77b: check CRR-N NRS policy ============== 15:50:45 (1776196245) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.4486 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0534186 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0803105 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163486 s, 6.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181983 s, 5.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17707 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0644435 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241176 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205605 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105145 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.293799 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216934 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240646 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.259373 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.33791 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255629 s, 4.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20475 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0702456 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00899077 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0403365 s, 26.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028415 s, 36.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007995 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.062794 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0869625 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00624543 s, 168 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650376 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0684018 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291362 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193425 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069348 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188031 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282345 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642732 s, 163 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.397986 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108898 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13503 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11524 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224532 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.27961 s, 3.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.329888 s, 3.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.297296 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.298647 s, 3.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.368343 s, 2.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.312894 s, 3.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.327549 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.387695 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.400601 s, 2.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.312719 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.348923 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.443366 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0473761 s, 22.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0521361 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0650627 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0380468 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0632524 s, 16.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.048193 s, 21.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0940097 s, 11.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0499038 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0728973 s, 14.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.060283 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0646013 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0470238 s, 22.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0589317 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0722214 s, 14.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0781429 s, 13.4 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0841428 s, 12.5 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (12s) == sanityn test 77c: check ORR NRS policy ================ 15:50:57 (1776196257) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.38789 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0682433 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0533671 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17716 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183831 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0920485 s, 11.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224789 s, 4.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0473995 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.270286 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216905 s, 4.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146998 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253915 s, 4.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.271247 s, 3.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.304434 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.287999 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.277304 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.329985 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0599579 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00322794 s, 325 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246594 s, 42.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757379 s, 138 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836353 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641011 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00461177 s, 227 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0588011 s, 17.8 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.070784 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0898355 s, 11.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0822682 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296916 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021978 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014833 s, 70.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115945 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022997 s, 45.6 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.544613 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0544975 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0587684 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103665 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164239 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162574 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179014 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166138 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244995 s, 4.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164953 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.367968 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.303447 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.247885 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.419768 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.347544 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.409464 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.375694 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0820763 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0829063 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0406483 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0890785 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0507718 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0506533 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0624998 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0828994 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0725108 s, 14.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0727344 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117971 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0721254 s, 14.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0821878 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0794017 s, 13.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.072213 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0893996 s, 11.7 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.492989 s, 34.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0655682 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0700658 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0879966 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138923 s, 7.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114999 s, 9.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14678 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152197 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0916589 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177591 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188806 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13261 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179384 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173118 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187941 s, 5.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178916 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121882 s, 8.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209558 s, 50.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0487828 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166205 s, 63.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134386 s, 78.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0637714 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0686214 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.095797 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0722626 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741648 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208978 s, 50.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0883712 s, 11.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00909379 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0028497 s, 368 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152695 s, 68.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993335 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00436002 s, 240 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (17s) == sanityn test 77d: check TRR nrs policy ================ 15:51:14 (1776196274) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.384077 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0830525 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104518 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147341 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181207 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255947 s, 4.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.29375 s, 3.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.311012 s, 3.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.304148 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.330151 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.26038 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.357538 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.384299 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.393017 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.403556 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.370266 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.387635 s, 2.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0561967 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0835564 s, 12.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.044424 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113079 s, 9.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122697 s, 8.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0578087 s, 18.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0579017 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0578686 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129359 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0675402 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.075298 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0788967 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0769232 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0856074 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0868452 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0940555 s, 11.1 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.367861 s, 45.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0798747 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0887066 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0646951 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.094605 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128179 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135193 s, 7.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161072 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216524 s, 4.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.213618 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232181 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.236754 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.226486 s, 4.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.250736 s, 4.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245419 s, 4.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.281454 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251846 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10838 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852121 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121608 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107389 s, 97.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135793 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00431208 s, 243 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103727 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0355432 s, 29.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124417 s, 8.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00616019 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733126 s, 143 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00313615 s, 334 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00459101 s, 228 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169821 s, 61.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109959 s, 95.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173737 s, 60.4 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.518467 s, 32.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0621288 s, 16.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0700543 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0729991 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233417 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127328 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0758011 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122631 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100838 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.142578 s, 7.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.179504 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134071 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156043 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108668 s, 9.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.258099 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.182556 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.292928 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0507493 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0756357 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0789079 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0626055 s, 16.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0807331 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0550314 s, 19.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0629826 s, 16.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0746189 s, 14.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1049 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0794703 s, 13.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0797015 s, 13.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0852176 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103826 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0918813 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111106 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121317 s, 8.6 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (15s) == sanityn test 77e: check TBF NID nrs policy ============ 15:51:29 (1776196289) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.203.14@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.405382 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0539292 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0680866 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14419 s, 7.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186841 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184742 s, 5.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.170987 s, 6.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184365 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169713 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.229656 s, 4.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.235665 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.236718 s, 4.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.244236 s, 4.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.225884 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22789 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.2591 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.263723 s, 4.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.252161 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.235914 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217864 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222745 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220875 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.224396 s, 4.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.235525 s, 4.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232753 s, 4.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.246601 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.297102 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.296457 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.254127 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260828 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.263548 s, 4.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248585 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.268549 s, 3.9 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.537916 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0500608 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0634999 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0694983 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117716 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154551 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144037 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.214722 s, 4.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192237 s, 5.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146421 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194654 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22909 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.29434 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.202362 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.287329 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260032 s, 4.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245003 s, 4.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0533877 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.055578 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0480349 s, 21.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0492421 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0773531 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0538839 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0580306 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0740974 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0668721 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0800533 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0822542 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0874177 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0916743 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0930483 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11057 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138344 s, 7.6 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.297235 s, 56.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143798 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16209 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134552 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.2301 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.235723 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181662 s, 5.8 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.28517 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.374478 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143677 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.395679 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.394445 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.418583 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.352609 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.376209 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.378104 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.409052 s, 2.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278505 s, 37.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140002 s, 7.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109149 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123792 s, 8.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0560072 s, 18.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110372 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158101 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118383 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155364 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176971 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144809 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133538 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168146 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137616 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0850054 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167755 s, 6.3 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.565874 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117769 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0748456 s, 14.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.23246 s, 4.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.181078 s, 5.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195665 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209273 s, 5.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168089 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139857 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.271651 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.421539 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.416239 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.315176 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.380228 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.482903 s, 2.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.492024 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.462852 s, 2.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154328 s, 67.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0638731 s, 16.4 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00268423 s, 391 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112937 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.231321 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.265089 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223505 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216802 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.177813 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.098251 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0874862 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191159 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0685589 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0460135 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145086 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0450954 s, 23.3 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (24s) == sanityn test 77f: check TBF JobID nrs policy ========== 15:51:54 (1776196314) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.273454 s, 61.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.040869 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0658669 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.060157 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0645016 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0727812 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155028 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190387 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22791 s, 4.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16928 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.220324 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.262121 s, 4.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.236768 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.24354 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.26196 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.272653 s, 3.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.301737 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00473243 s, 222 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0580562 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00328377 s, 319 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0397407 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00602541 s, 174 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111371 s, 94.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120654 s, 86.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00930629 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131201 s, 79.9 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00512982 s, 204 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0709923 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993332 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0462646 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132146 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0953379 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113659 s, 92.3 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.441374 s, 38.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0586742 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134014 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.166373 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153001 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101211 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151857 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.24194 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.282361 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.192582 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20851 s, 5.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.295037 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.253853 s, 4.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251858 s, 4.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.3212 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.284308 s, 3.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.258331 s, 4.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00573276 s, 183 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00440696 s, 238 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133895 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00378177 s, 277 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129741 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0094758 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127732 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0893916 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0916991 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00506203 s, 207 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154311 s, 6.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0591702 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0734164 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0753056 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0500827 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0441054 s, 23.8 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.437694 s, 38.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0409117 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0569059 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132383 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0939453 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156744 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.178586 s, 5.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168662 s, 6.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.219157 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207774 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167877 s, 6.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.197233 s, 5.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148697 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130198 s, 8.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187986 s, 5.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218021 s, 4.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.21654 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0428298 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00327055 s, 321 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271301 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00215783 s, 486 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107066 s, 97.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684934 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0811469 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117649 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00263187 s, 398 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0942543 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.029462 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823538 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0536703 s, 19.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248189 s, 42.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250617 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256247 s, 40.9 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.407617 s, 41.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0784094 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0764581 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0840685 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209137 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.231463 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22043 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240911 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251062 s, 4.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.27094 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.272904 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.261029 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.29521 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.249256 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.305713 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.28644 s, 3.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.269322 s, 3.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0620083 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0442317 s, 23.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.056719 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.065647 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0549065 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.062589 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0901388 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0698147 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0667666 s, 15.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0753599 s, 13.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0585661 s, 17.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.073401 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0665171 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.078301 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0741123 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0785106 s, 13.4 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (24s) == sanityn test 77g: Change TBF type directly ============ 15:52:17 (1776196337) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (12s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 15:52:30 (1776196350) oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg314-client: oleg314-server: ssh exited with exit code 2 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg314-client: oleg314-server: ssh exited with exit code 12 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg314-client: oleg314-server: ssh exited with exit code 12 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg314-client: oleg314-server: ssh exited with exit code 12 PASS 77h (15s) == sanityn test 77i: Change rank of TBF rule ============= 15:52:46 (1776196366) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (24s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 15:53:10 (1776196390) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.3995 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0721375 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114294 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104644 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109503 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.088553 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132483 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122603 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.099904 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101486 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20593 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.202507 s, 5.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203896 s, 5.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198958 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.225481 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.205898 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.248996 s, 4.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251066 s, 41.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030804 s, 34.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0360407 s, 29.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0669641 s, 15.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0377205 s, 27.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372054 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0438551 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0605159 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0961572 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.055884 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0471986 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0522881 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0605026 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0673274 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0665604 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0689546 s, 15.2 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 7.8507 s, 13.4 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.613 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (67s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 15:54:16 (1776196456) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.887535 s, 18.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0706465 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12755 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137262 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107754 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188874 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.2068 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147587 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.22656 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.26305 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.257023 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260495 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.294318 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.324329 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.333119 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.362415 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.419212 s, 2.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0726369 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0764116 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.06748 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0964101 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0311053 s, 33.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0487623 s, 21.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0777126 s, 13.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.074074 s, 14.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0718048 s, 14.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242015 s, 43.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0808485 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0895347 s, 11.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0924051 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107463 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138056 s, 76.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0041977 s, 250 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8276 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9608 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.820087 s, 20.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136196 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151974 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157486 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.221807 s, 4.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.216977 s, 4.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.201821 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.275471 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.267283 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.303308 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.330331 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.356499 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.387056 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.406178 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.422561 s, 2.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.429162 s, 2.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.444044 s, 2.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305327 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112792 s, 93.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0650018 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176117 s, 59.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197632 s, 53.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202242 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022233 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309272 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0404641 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0521542 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0778102 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0798486 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0458021 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0448854 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.073502 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0506745 s, 20.7 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.032 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9477 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (149s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 15:56:45 (1776196605) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.588802 s, 28.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.041934 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0616961 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0822172 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156248 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162168 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.20003 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.251435 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218946 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223381 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.265594 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.282607 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.273329 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.306863 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.284495 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.306899 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.325487 s, 3.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=3 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860329 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0700536 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0669138 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.096118 s, 10.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0744138 s, 14.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104807 s, 10.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0923692 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0985938 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.092941 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108244 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0924895 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0925311 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0979827 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0963954 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0991145 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111374 s, 9.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.85682 s, 10.6 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.95683 s, 10.5 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.14@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.14@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.583396 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0820269 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0829911 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0735806 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0771684 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.176001 s, 6.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185847 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.270753 s, 3.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.217583 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.268552 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.335088 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.294932 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.365275 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.312496 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.334284 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.376445 s, 2.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff964048299800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.341556 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.163084 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156951 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194516 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.16538 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208444 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.184992 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200001 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1835 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.187784 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.219449 s, 4.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203064 s, 5.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222841 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.208342 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.209593 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.212893 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.218069 s, 4.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.44902 s, 11.1 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.93399 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.14@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.460034 s, 36.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0632008 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0490314 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0806714 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0882538 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.219168 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.164989 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.180073 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.241219 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.231547 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203107 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.249586 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.2764 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.27795 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.211113 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.277458 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.235826 s, 4.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0594878 s, 17.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137266 s, 76.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0383967 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0393495 s, 26.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0529107 s, 19.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0417979 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0463279 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145764 s, 71.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0536012 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00547665 s, 191 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835231 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0628491 s, 16.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100944 s, 10.4 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0692659 s, 15.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0653646 s, 16.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117932 s, 8.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 7.74693 s, 13.5 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.0534 s, 20.7 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.544205 s, 30.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0751521 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104801 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169975 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.198011 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.200311 s, 5.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.238064 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.284616 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.232318 s, 4.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.28868 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.277467 s, 3.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.284249 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.257337 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.26315 s, 4.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.323755 s, 3.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.255376 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.268384 s, 3.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130986 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0917135 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123164 s, 85.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012532 s, 83.7 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743519 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113853 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00470968 s, 223 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0048277 s, 217 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0947727 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115708 s, 9.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117953 s, 88.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0748863 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0803925 s, 13.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189486 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122339 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0593873 s, 17.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.6512 s, 9.8 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.91925 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 7.82655 s, 13.4 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.90929 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.846634 s, 19.8 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.045874 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0763153 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111324 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112775 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150883 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162985 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127098 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159119 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260832 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317676 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.279982 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.369298 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.386334 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.40467 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.406419 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.411899 s, 2.5 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319641 s, 32.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0661401 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0583017 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0948853 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167009 s, 62.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10318 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0967453 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113952 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0987539 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121542 s, 8.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0573264 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102399 s, 102 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125589 s, 8.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106506 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0752352 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136857 s, 7.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 20.1208 s, 5.2 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6409 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.80803 s, 20.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0661996 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0891098 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.167981 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106358 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0966574 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.183439 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.185694 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134304 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186988 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.219476 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.222411 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1784 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130522 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.349012 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.403776 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.524827 s, 2.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298783 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0600946 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0410494 s, 25.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0581908 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0997972 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0691698 s, 15.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0775013 s, 13.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0717438 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130287 s, 8.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0930165 s, 11.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0987917 s, 10.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100752 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114171 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107045 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109582 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116236 s, 9.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg314-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 12.882 s, 8.1 MB/s Write runtime is 14 s, speed is 7.142857 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.92912 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.518181 s, 32.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0763994 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0606009 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117852 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0739293 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104687 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.048377 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14957 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10908 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0586488 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0768006 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.169802 s, 6.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11765 s, 8.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203641 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.188666 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17032 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.267189 s, 3.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381942 s, 27.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0701795 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.044827 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0437908 s, 23.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0644258 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0708184 s, 14.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0581483 s, 18.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0575399 s, 18.2 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0673894 s, 15.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0646595 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0785192 s, 13.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0759775 s, 13.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0677379 s, 15.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.073607 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0975384 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0727995 s, 14.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 11.3286 s, 9.3 MB/s Write runtime is 13 s, speed is 7.692307 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.91502 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 10.6852 s, 9.8 MB/s Write runtime is 12 s, speed is 8.333333 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.92815 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (435s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 16:04:00 (1776197040) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (12s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 16:04:12 (1776197052) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg314-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.92403 s, 267 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg314-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.0754 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (57s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 16:05:10 (1776197110) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.470619 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0963486 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145182 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0998122 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11776 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111433 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143097 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0896872 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0871677 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174277 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.28851 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.336232 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.341692 s, 3.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.336109 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.321605 s, 3.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317839 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.322838 s, 3.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff964048299800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272901 s, 38.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0669837 s, 15.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0386845 s, 27.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0885623 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0661456 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0844654 s, 12.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0606262 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0553553 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0930816 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0916486 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111026 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0831332 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0739383 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.092484 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0925381 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0874912 s, 12.0 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.8592 s, 11.8 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.24661 s, 20.0 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.402094 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0584701 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0760714 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101692 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0973107 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113222 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134665 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.190429 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.207671 s, 5.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245164 s, 4.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.239617 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.233655 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240977 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.249783 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.2364 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.236529 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.270948 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776186 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0433717 s, 24.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0761529 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0482494 s, 21.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0634604 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0526827 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0593593 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.032711 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0512713 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132618 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0932586 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132097 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109069 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218863 s, 47.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281068 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126548 s, 8.3 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg314-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg314-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.73006 s, 12.0 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.15921 s, 20.3 MB/s Read runtime is 7 s, speed is 14.285714 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (90s) == sanityn test 77o: Changing rank should not panic ====== 16:06:39 (1776197199) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (12s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 16:06:51 (1776197211) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (109s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 16:08:40 (1776197320) mds.MDS.mdt.nrs_policies=tbf oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg314-client: oleg314-server: ssh exited with exit code 36 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg314-client: oleg314-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (34s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 16:09:14 (1776197354) oleg314-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: Resource temporarily unavailable pdsh@oleg314-client: oleg314-server: ssh exited with exit code 11 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (9s) == sanityn test 79: xattr: intent error ================== 16:09:23 (1776197363) File: /mnt/lustre/d79.sanityn Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505430 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-04-14 16:09:25.000000000 -0400 Modify: 2026-04-14 16:09:25.000000000 -0400 Change: 2026-04-14 16:09:25.000000000 -0400 Birth: 2026-04-14 16:09:26.000000000 -0400 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (18s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 16:09:42 (1776197382) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (3s) == sanityn test 80b: Accessing directory during migration ========================================================== 16:09:44 (1776197384) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (4s) == sanityn test 81a: rename and stat under striped directory ========================================================== 16:09:49 (1776197389) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (6s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 16:09:54 (1776197394) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (6s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 16:10:00 (1776197400) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (4s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 16:10:04 (1776197404) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7142 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7142 PASS 82 (8s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 16:10:12 (1776197412) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (4s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 16:10:16 (1776197416) fail_loc=0x8000060b PASS 84 (15s) == sanityn test 90: open/create and unlink striped directory ========================================================== 16:10:31 (1776197431) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (4s) == sanityn test 91: chmod and unlink striped directory === 16:10:35 (1776197435) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (4s) == sanityn test 92: create remote directory under orphan directory ========================================================== 16:10:39 (1776197439) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (4s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 16:10:43 (1776197443) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2642 0xa52 0 1 2628 0xa44 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2643 0xa53 0 1 2629 0xa45 0 rc1=0 and rc2=0 PASS 93 (17s) == sanityn test 94: signal vs CP callback race =========== 16:11:00 (1776197460) osc.lustre-OST0000-osc-ffff964048299800.idle_timeout=0 osc.lustre-OST0000-osc-ffff964060cdb800.idle_timeout=0 osc.lustre-OST0001-osc-ffff964048299800.idle_timeout=0 osc.lustre-OST0001-osc-ffff964060cdb800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.223426 s, 36.7 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 3.01299 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 282579 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.110186 s, 37.2 kB/s osc.lustre-OST0000-osc-ffff964048299800.idle_timeout=debug osc.lustre-OST0000-osc-ffff964060cdb800.idle_timeout=debug osc.lustre-OST0001-osc-ffff964048299800.idle_timeout=debug osc.lustre-OST0001-osc-ffff964060cdb800.idle_timeout=debug PASS 94 (16s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 16:11:16 (1776197476) llite.lustre-ffff964048299800.fast_read=0 llite.lustre-ffff964060cdb800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.00899536 s, 1.4 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff964048299800.fast_read=1 llite.lustre-ffff964060cdb800.fast_read=1 PASS 95b (13s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 16:11:29 (1776197489) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (3s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 16:11:32 (1776197492) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00909206 s, 451 kB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (7s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 16:11:39 (1776197499) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.154762 s, 13.6 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (7s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 16:11:47 (1776197507) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.11564 s, 18.1 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (7s) == sanityn test 100e: DoM: read on open and file size ==== 16:11:53 (1776197513) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.00970765 s, 1.2 MB/s PASS 100e (8s) == sanityn test 101a: Discard DoM data on unlink ========= 16:12:01 (1776197521) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00564118 s, 726 kB/s PASS 101a (7s) == sanityn test 101b: Discard DoM data on rename ========= 16:12:08 (1776197528) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00671276 s, 610 kB/s PASS 101b (7s) == sanityn test 101c: Discard DoM data on close-unlink === 16:12:15 (1776197535) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00632218 s, 648 kB/s PASS 101c (8s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 16:12:24 (1776197544) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1776197547 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1776197551 Finishing test test23 at 1776197551 PASS 103 (14s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 16:12:38 (1776197558) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (3s) == sanityn test 105: Glimpse and lock cancel race ======== 16:12:42 (1776197562) fail_loc=0x416 PASS 105 (53s) == sanityn test 106a: Verify the btime via statx() ======= 16:13:34 (1776197614) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (3s) == sanityn test 106b: Glimpse RPCs test for statx ======== 16:13:37 (1776197617) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0159675 s, 64.1 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505458 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-04-14 16:13:39.000000000 -0400 Modify: 2026-04-14 16:13:39.000000000 -0400 Change: 2026-04-14 16:13:39.000000000 -0400 Birth: 2026-04-14 16:13:40.000000000 -0400 /mnt/lustre/f106b.sanityn 144115205272505458 -rw-r--r-- 1 0 0 1776197620 1776197619 1776197619 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505458 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-04-14 16:13:39.000000000 -0400 Modify: 2026-04-14 16:13:39.000000000 -0400 Change: 2026-04-14 16:13:39.000000000 -0400 Birth: 2026-04-14 16:13:40.000000000 -0400 1776197619 1024 1 PASS 106b (8s) == sanityn test 106c: Verify statx attributes mask ======= 16:13:45 (1776197625) PASS 106c (8s) == sanityn test 107a: Basic grouplock conflict =========== 16:13:53 (1776197633) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.269005 s, 39.0 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2651 0xa5b 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 First grouplock blocks second one PASS 107a (8s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 16:14:01 (1776197641) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.425321 s, 24.7 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2634 0xa4a 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (15s) == sanityn test 108a: lseek: parallel updates ============ 16:14:16 (1776197656) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.00697528 s, 1.4 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 4.94559 s, 1.7 MB/s PASS 108a (13s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 16:14:30 (1776197670) 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 192.168.203.114@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.114@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg314-server' oleg314-server: oleg314-server.virtnet: executing load_modules_local oleg314-server: Loading modules from /home/green/git/lustre-release/lustre oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg314-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg314-server: detected 4 online CPUs by sysfs oleg314-server: Force libcfs to create 2 CPU partitions Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre Starting client: oleg314-client.virtnet: -o user_xattr,flock oleg314-server@tcp:/lustre /mnt/lustre2 PASS 109 (1204s) == sanityn test 110: do not grant another lock on resend ========================================================== 16:34:34 (1776198874) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-04-14 16:34:35.000000000 -0400 Modify: 2026-04-14 16:34:35.000000000 -0400 Change: 2026-04-14 16:34:35.000000000 -0400 Birth: 2026-04-14 16:34:36.000000000 -0400 fail_loc=0 fail_val=0 Connected clients: oleg314-client.virtnet oleg314-client.virtnet PASS 110 (42s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 16:35:16 (1776198916) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (3s) == sanityn test 112: update max-inherit in default LMV === 16:35:19 (1776198919) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (3s) == sanityn test 113: check servers of specified fs ======= 16:35:22 (1776198922) lustre-OST0000-osc-ffff964045188000 active. lustre-OST0001-osc-ffff964045188000 active. lustre-OST0000-osc-ffff964045188000 active. lustre-OST0001-osc-ffff964045188000 active. MGC192.168.203.114@tcp active. PASS 113 (7s) cleanup: ====================================================== == sanityn test complete, duration 9015 sec ============== 16:35:31 (1776198931) Stopping clients: oleg314-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg314-client.virtnet /mnt/lustre2 opts: