-----============= acceptance-small: sanityn ============----- Tue Jun 9 19:06:56 EDT 2026 oleg305-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Started clients oleg305-client.virtnet: 192.168.203.105@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt) oleg305-client.virtnet: executing check_config_client /mnt/lustre oleg305-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg305-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff975f46293000.idle_timeout=debug osc.lustre-OST0000-osc-ffff975f48093800.idle_timeout=debug osc.lustre-OST0001-osc-ffff975f46293000.idle_timeout=debug osc.lustre-OST0001-osc-ffff975f48093800.idle_timeout=debug disable quota as required oleg305-server: oleg305-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124723 s, 84.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f7129] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 19:07:09 (1781046429) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (2s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 19:07:11 (1781046431) -rw-r--r-- 1 root root 0 Jun 9 19:07 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (2s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 19:07:13 (1781046433) -rw-r--r-- 1 root root 0 Jun 9 19:07 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (2s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 19:07:15 (1781046435) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (2s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 19:07:17 (1781046437) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (1s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 19:07:19 (1781046439) total 1 -rwxrwxrwx 1 root root 0 Jun 9 19:07 f2a -rwxrwxrwx 1 root root 0 Jun 9 19:07 f2b total 1 -rwxrwxrwx 1 root root 0 Jun 9 19:07 f2a -rwxrwxrwx 1 root root 0 Jun 9 19:07 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.00229941 s, 223 kB/s PASS 2e (1s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 19:07:20 (1781046440) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 19:07:21 (1781046441) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0745947 s, 28.1 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (2s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 19:07:23 (1781046443) PASS 3 (2s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 19:07:25 (1781046445) PASS 4 (3s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 19:07:28 (1781046448) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (1s) == sanityn test 6: remove of open file on other node ============================================================================ 19:07:30 (1781046450) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (1s) == sanityn test 7: remove of open directory on other node ======================================================================= 19:07:31 (1781046451) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (3s) == sanityn test 8: remove of open special file on other node ==================================================================== 19:07:34 (1781046454) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (2s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 19:07:36 (1781046456) PASS 9a (2s) == sanityn test 9b: append to striped sparse file ======== 19:07:38 (1781046458) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015283 s, 68.6 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.0106768 s, 0.3 kB/s Data read (expecting 'foo'): foo PASS 9b (2s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 19:07:40 (1781046460) 1+0 records in 1+0 records out 1 byte copied, 0.00185468 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00152729 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00141833 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00200903 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00162034 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00147808 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00196608 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00159514 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00179602 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00151932 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00137299 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00145017 s, 0.7 kB/s PASS 10a (2s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 19:07:42 (1781046462) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.0025145 s, 1.2 MB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00744365 s, 550 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.000198066 s, 15.5 MB/s PASS 10b (2s) == sanityn test 11: execution of file opened for write should return error ============================================================== 19:07:44 (1781046464) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (2s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 19:07:46 (1781046466) warning: '-runtime' deprecated, use '-t runtime' instead - link 2602 (time 1781046477.17 total 10.00 last 260.13) - link 5000 (time 1781046487.18 total 20.00 last 239.76) - link 7375 (time 1781046497.18 total 30.01 last 237.41) - link 9770 (time 1781046507.18 total 40.01 last 239.46) - link 12214 (time 1781046517.18 total 50.01 last 244.35) - link 14360 (time 1781046527.19 total 60.02 last 214.48) - link 16579 (time 1781046537.19 total 70.02 last 221.89) - link 18878 (time 1781046547.19 total 80.02 last 229.82) - link 20000 (time 1781046552.15 total 84.97 last 226.48) - link 22397 (time 1781046562.15 total 94.98 last 239.62) total: 23574 link in 100.00 seconds: 235.73 ops/second using seed 2950213117 running for 100 seconds - stat 10000 (time 1781046472 ; total 4 ; last 4) - stat 20000 (time 1781046476 ; total 8 ; last 4) - stat 30000 (time 1781046480 ; total 12 ; last 4) - stat 40000 (time 1781046484 ; total 16 ; last 4) - stat 50000 (time 1781046489 ; total 21 ; last 5) - stat 60000 (time 1781046493 ; total 25 ; last 4) - stat 70000 (time 1781046497 ; total 29 ; last 4) - stat 80000 (time 1781046502 ; total 34 ; last 5) - stat 90000 (time 1781046506 ; total 38 ; last 4) - stat 100000 (time 1781046510 ; total 42 ; last 4) - stat 110000 (time 1781046514 ; total 46 ; last 4) - stat 120000 (time 1781046520 ; total 52 ; last 6) - stat 129187 (time 1781046531 ; total 63 ; last 11) - stat 130000 (time 1781046531 ; total 63 ; last 0) - stat 139750 (time 1781046542 ; total 74 ; last 11) - stat 140000 (time 1781046542 ; total 74 ; last 0) - stat 150000 (time 1781046552 ; total 84 ; last 10) - stat 160000 (time 1781046562 ; total 94 ; last 10) total: 166848 stats in 100 seconds: 1668.479980 stats/second - unlinked 0 (time 1781046518 ; total 0 ; last 0) - unlinked 10000 (time 1781046573 ; total 55 ; last 55) - unlinked 20000 (time 1781046598 ; total 80 ; last 25) unlink(/mnt/lustre2/lockdir/lockfile23574) error: No such file or directory total: 23573 unlinks in 90 seconds: 261.922211 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (19681) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (19683) - No such process PASS 12 (143s) == sanityn test 13: test directory page revocation ======= 19:10:10 (1781046610) f13 f13 PASS 13 (2s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 19:10:12 (1781046612) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 20853 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 19:10:14 (1781046614) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 21425 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (2s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 19:10:16 (1781046616) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 21997 Terminated $DIR1/$tdir/sleep 60 PASS 14b (2s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 19:10:18 (1781046618) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 22572 Terminated $DIR1/$tdir/sleep 60 PASS 14c (2s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 19:10:20 (1781046620) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 23147 Terminated $DIR1/$tdir/sleep 60 PASS 14d (3s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 19:10:23 (1781046623) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7519232 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7519232kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (11s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 19:10:34 (1781046634) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0766321 s, 27.4 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0583075 s, 36.0 MB/s Chance of close/open is 1 in 50 Seed set to 6635 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[0] 1781046636.140795 write 0x82f23f thru 0x83c642 (0xd404 bytes) 000003[1] 1781046636.146708 mapread 0x789708 thru 0x78bdcf (0x26c8 bytes) 000005[0] 1781046636.168185 read 0x2fb8ad thru 0x302904 (0x7058 bytes) 000006[0] 1781046636.175555 mapread 0x6e6813 thru 0x6e9594 (0x2d82 bytes) 000007[1] 1781046636.180805 mapread 0x129cac thru 0x132758 (0x8aad bytes) 000008[0] 1781046636.225842 write 0x1584a9 thru 0x15c2c1 (0x3e19 bytes) 000009[1] 1781046636.237457 write 0x0ef8cf thru 0x0efda7 (0x4d9 bytes) truncating to largest ever: 0x6dcdd1 000010[1] 1781046636.250074 trunc from 0x83c643 to 0x6dcdd0 (0x15f872 bytes) 000012[1] 1781046636.266092 mapwrite 0x0bf5a8 thru 0x0c1ea3 (0x28fc bytes) truncating to largest ever: 0x7d3906 000013[1] 1781046636.299310 trunc from 0x6dcdd1 to 0x7d3905 (0xf6b35 bytes) 000014[0] 1781046636.305847 write 0x790b90 thru 0x79208c (0x14fd bytes) 000015[1] 1781046636.315027 trunc from 0x7d3906 to 0x4ed344 (0x2e65c1 bytes) 000016[1] 1781046636.327204 read 0x1c4828 thru 0x1ca09c (0x5875 bytes) 000017[0] 1781046636.333752 mapread 0x20df3d thru 0x217074 (0x9138 bytes) 000019[0] 1781046636.349615 mapwrite 0x0a7a70 thru 0x0b6655 (0xebe6 bytes) 000022[1] 1781046636.388577 mapread 0x064031 thru 0x06951e (0x54ee bytes) 000023[0] 1781046636.401941 trunc from 0x4ed345 to 0x3920c3 (0x15b281 bytes) 000024[1] 1781046636.413215 write 0x3ba785 thru 0x3bf75a (0x4fd6 bytes) truncating to largest ever: 0x7f3e4e 000025[1] 1781046636.419667 trunc from 0x3bf75b to 0x7f3e4d (0x4346f3 bytes) 000026[0] 1781046636.433771 mapwrite 0x09da89 thru 0x0a80cb (0xa643 bytes) 000027[0] 1781046636.462152 mapread 0x34f7e7 thru 0x359666 (0x9e80 bytes) 000028[0] 1781046636.479764 read 0x32bd39 thru 0x32c575 (0x83d bytes) 000029[1] 1781046636.511779 trunc from 0x7f3e4e to 0x43d9b4 (0x3b6499 bytes) 000030[0] 1781046636.531138 read 0x19ae1e thru 0x1a523e (0xa421 bytes) 000032[0] 1781046636.544754 trunc from 0x43d9b5 to 0x46d948 (0x2ff94 bytes) truncating to largest ever: 0x8dd540 000033[0] 1781046636.553191 trunc from 0x46d949 to 0x8dd53f (0x46fbf7 bytes) 000035[0] 1781046636.557820 mapread 0x21f2a5 thru 0x220712 (0x146e bytes) 000036[1] 1781046636.587157 read 0x45388d thru 0x4567d2 (0x2f46 bytes) 000037[0] 1781046636.613950 mapwrite 0x6c9d0f thru 0x6d2c85 (0x8f77 bytes) 000039[0] 1781046636.646344 trunc from 0x8dd540 to 0x75fe40 (0x17d6ff bytes) 000040[1] 1781046636.655179 mapwrite 0x6ffc1b thru 0x701278 (0x165e bytes) 000042[0] 1781046636.703430 mapread 0x4a765d thru 0x4aab8c (0x3530 bytes) 000043[1] 1781046636.716823 write 0x3df2f5 thru 0x3e5e49 (0x6b55 bytes) 000049[0] 1781046636.731922 write 0x934405 thru 0x9418cf (0xd4cb bytes) 000050[0] 1781046636.741989 read 0x0a66a7 thru 0x0b1038 (0xa992 bytes) 000051[1] 1781046636.781332 read 0x0ef590 thru 0x0f66a5 (0x7116 bytes) 000052[0] 1781046636.788899 write 0x8a4662 thru 0x8b0bbe (0xc55d bytes) 000053[1] 1781046636.805601 trunc from 0x9418d0 to 0x77e756 (0x1c3179 bytes) 000057[1] 1781046636.829608 mapread 0x5c7cbc thru 0x5c95fd (0x1942 bytes) 000059[1] 1781046636.870463 write 0x8c0c17 thru 0x8c3505 (0x28ef bytes) 000061[1] 1781046636.876543 mapread 0x18c9b7 thru 0x196ac6 (0xa110 bytes) 000062[0] 1781046636.881249 mapwrite 0x408f2f thru 0x40b102 (0x21d4 bytes) 000063[1] 1781046636.908928 mapread 0x57e66b thru 0x58c0e0 (0xda76 bytes) 000065[1] 1781046636.917934 write 0x18e209 thru 0x198a2a (0xa822 bytes) 000067[0] 1781046636.925232 mapwrite 0x74d976 thru 0x755176 (0x7801 bytes) 000070[0] 1781046636.963039 write 0x40f417 thru 0x4158ab (0x6495 bytes) 000073[1] 1781046636.976847 mapread 0x12d00f thru 0x1397fd (0xc7ef bytes) 000074[0] 1781046636.994847 trunc from 0x8c3506 to 0x15ea98 (0x764a6d bytes) 000075[0] 1781046637.006203 read 0x09bd5d thru 0x0a1fb4 (0x6258 bytes) 000076[1] 1781046637.014515 mapread 0x11b7b4 thru 0x12a3ce (0xec1b bytes) 000077[1] 1781046637.037981 mapwrite 0x8d78ff thru 0x8dabac (0x32ae bytes) 000079[0] 1781046637.079189 trunc from 0x8dabad to 0x8d941c (0x1790 bytes) 000081[1] 1781046637.089163 write 0x904881 thru 0x911447 (0xcbc7 bytes) 000082[1] 1781046637.096117 mapread 0x704cb6 thru 0x70a420 (0x576b bytes) 000083[1] 1781046637.108605 read 0x377cfd thru 0x381d87 (0xa08b bytes) 000084[0] 1781046637.117126 write 0x577571 thru 0x57d376 (0x5e06 bytes) 000088[0] 1781046637.178145 mapwrite 0x8c3b97 thru 0x8c3ce3 (0x14d bytes) 000090[1] 1781046637.192691 trunc from 0x911448 to 0x01c5e3 (0x8f4e64 bytes) 000091[1] 1781046637.213951 mapread 0x016aef thru 0x01c5e3 (0x5af5 bytes) 000092[0] 1781046637.222353 read 0x017fe4 thru 0x01c5e3 (0x4600 bytes) 000093[0] 1781046637.227354 mapread 0x00dfa5 thru 0x014e74 (0x6ed0 bytes) 000094[0] 1781046637.238867 write 0x53243b thru 0x538413 (0x5fd9 bytes) 000098[1] 1781046637.258358 mapwrite 0x0c8c67 thru 0x0d12ce (0x8668 bytes) 000099[1] 1781046637.325375 mapread 0x2e2038 thru 0x2eb58b (0x9554 bytes) 000100[1] 1781046637.357710 mapread 0x00f409 thru 0x01788c (0x8484 bytes) 000102[0] 1781046637.375645 trunc from 0x538414 to 0x267e39 (0x2d05da bytes) 000104[1] 1781046637.386741 read 0x09ea96 thru 0x0acd65 (0xe2d0 bytes) 000106[1] 1781046637.397867 read 0x203590 thru 0x20ff1b (0xc98c bytes) 000108[1] 1781046637.409595 write 0x523b03 thru 0x52e413 (0xa911 bytes) 000109[0] 1781046637.416460 write 0x0c61fb thru 0x0d2daf (0xcbb5 bytes) 000110[1] 1781046637.430883 read 0x02361c thru 0x0283a2 (0x4d87 bytes) 000111[1] 1781046637.467472 read 0x1ea5a6 thru 0x1eb367 (0xdc2 bytes) truncating to largest ever: 0x9f92e6 000112[1] 1781046637.467893 trunc from 0x52e414 to 0x9f92e5 (0x4caed2 bytes) 000113[0] 1781046637.478380 mapread 0x00a595 thru 0x01865a (0xe0c6 bytes) 000116[1] 1781046637.506647 mapwrite 0x838dc8 thru 0x83b99d (0x2bd6 bytes) 000119[0] 1781046637.530054 mapwrite 0x121840 thru 0x12379f (0x1f60 bytes) 000121[0] 1781046637.566375 read 0x2e445d thru 0x2edcab (0x984f bytes) 000124[0] 1781046637.617901 mapwrite 0x718cef thru 0x71fe86 (0x7198 bytes) 000125[1] 1781046637.631582 mapread 0x6b385d thru 0x6b83cc (0x4b70 bytes) 000126[0] 1781046637.642906 mapwrite 0x3d5ef9 thru 0x3e3b3b (0xdc43 bytes) 000127[0] 1781046637.660849 mapwrite 0x4cc05e thru 0x4d2b74 (0x6b17 bytes) 000128[0] 1781046637.698930 read 0x65a00c thru 0x65c085 (0x207a bytes) 000129[0] 1781046637.718533 trunc from 0x9f92e6 to 0x893d37 (0x1655ae bytes) 000130[1] 1781046637.725460 mapread 0x7718e6 thru 0x77faf3 (0xe20e bytes) 000137[1] 1781046637.768881 read 0x32e757 thru 0x32ec49 (0x4f3 bytes) 000138[0] 1781046637.777404 mapwrite 0x8f3b7a thru 0x8f5723 (0x1baa bytes) 000139[0] 1781046637.804874 mapread 0x45f5cb thru 0x4687f1 (0x9227 bytes) 000140[1] 1781046637.819581 write 0x724532 thru 0x726d08 (0x27d7 bytes) 000141[0] 1781046637.827470 mapread 0x7dfdfc thru 0x7e7f51 (0x8156 bytes) 000143[1] 1781046637.846856 read 0x3c3ee0 thru 0x3cb3ca (0x74eb bytes) 000146[1] 1781046637.858224 mapread 0x33e909 thru 0x340946 (0x203e bytes) 000147[1] 1781046637.866840 mapwrite 0x04984c thru 0x0520a0 (0x8855 bytes) 000148[1] 1781046637.928272 mapwrite 0x95e07c thru 0x967505 (0x948a bytes) 000150[0] 1781046637.962471 mapwrite 0x0de22f thru 0x0e5f9d (0x7d6f bytes) 000151[1] 1781046637.995375 trunc from 0x967506 to 0x9a2263 (0x3ad5e bytes) 000153[0] 1781046638.007350 read 0x7e4d93 thru 0x7f3170 (0xe3de bytes) 000154[1] 1781046638.015615 read 0x2516de thru 0x2605ff (0xef22 bytes) 000155[0] 1781046638.024949 trunc from 0x9a2264 to 0x6d468b (0x2cdbd8 bytes) 000156[0] 1781046638.034370 read 0x08a9b8 thru 0x0916f7 (0x6d40 bytes) 000157[1] 1781046638.071488 write 0x21ba57 thru 0x21e644 (0x2bee bytes) 000161[0] 1781046638.091692 mapread 0x3daa44 thru 0x3e216c (0x7729 bytes) 000163[1] 1781046638.100675 read 0x30ecea thru 0x31b4f8 (0xc80f bytes) 000165[0] 1781046638.135408 write 0x6d3569 thru 0x6da346 (0x6dde bytes) 000168[0] 1781046638.148507 mapread 0x404162 thru 0x412285 (0xe124 bytes) 000174[1] 1781046638.176421 write 0x6e536f thru 0x6eb9f3 (0x6685 bytes) 000176[1] 1781046638.188059 mapwrite 0x3e4d8f thru 0x3ec22d (0x749f bytes) 000180[0] 1781046638.239980 write 0x393556 thru 0x3a3267 (0xfd12 bytes) 000181[1] 1781046638.248210 read 0x1899a7 thru 0x193f90 (0xa5ea bytes) 000182[0] 1781046638.275386 mapread 0x2cf0bb thru 0x2d15e4 (0x252a bytes) 000184[1] 1781046638.289448 read 0x1a0de8 thru 0x1af429 (0xe642 bytes) 000186[1] 1781046638.291455 trunc from 0x6eb9f4 to 0x0f6f0c (0x5f4ae7 bytes) 000189[1] 1781046638.314722 trunc from 0x0f6f0d to 0x34711f (0x250213 bytes) 000190[0] 1781046638.323525 read 0x1df258 thru 0x1ec3ad (0xd156 bytes) 000191[0] 1781046638.331470 mapread 0x260551 thru 0x26a9cf (0xa47f bytes) 000192[1] 1781046638.346185 read 0x0e8421 thru 0x0ec096 (0x3c76 bytes) 000193[0] 1781046638.352236 read 0x004e13 thru 0x00fa31 (0xac1f bytes) 000194[1] 1781046638.368653 trunc from 0x347120 to 0x4d54e3 (0x18e3c4 bytes) 000197[1] 1781046638.391153 mapwrite 0x6e9b01 thru 0x6f4595 (0xaa95 bytes) 000198[1] 1781046638.446368 read 0x586541 thru 0x58b36b (0x4e2b bytes) 000199[0] 1781046638.446758 mapread 0x1a75f6 thru 0x1b6df4 (0xf7ff bytes) 000200[1] 1781046638.478542 mapwrite 0x444055 thru 0x445431 (0x13dd bytes) 000202[0] 1781046638.497249 read 0x0e2732 thru 0x0e986f (0x713e bytes) 000208[0] 1781046638.537907 mapwrite 0x6298bb thru 0x630dd3 (0x7519 bytes) 000209[0] 1781046638.578624 mapwrite 0x05d8be thru 0x06ce36 (0xf579 bytes) 000210[0] 1781046638.613296 mapwrite 0x18e784 thru 0x192d4c (0x45c9 bytes) 000211[0] 1781046638.653390 trunc from 0x6f4596 to 0x2938c4 (0x460cd1 bytes) 000212[1] 1781046638.661882 write 0x953bc4 thru 0x963948 (0xfd85 bytes) 000213[1] 1781046638.671026 mapread 0x206d66 thru 0x21589f (0xeb3a bytes) 000214[1] 1781046638.694237 trunc from 0x963949 to 0x83aed7 (0x128a71 bytes) 000216[0] 1781046638.706380 read 0x09d9cb thru 0x0a7842 (0x9e78 bytes) 000217[0] 1781046638.715557 read 0x5b1af3 thru 0x5b389d (0x1dab bytes) 000219[1] 1781046638.744576 read 0x7d00dd thru 0x7d61ec (0x6110 bytes) 000221[0] 1781046638.751033 mapwrite 0x344b20 thru 0x34b2ed (0x67ce bytes) 000222[0] 1781046638.771614 mapread 0x4b6240 thru 0x4b8151 (0x1f12 bytes) 000229[0] 1781046638.782149 mapread 0x622170 thru 0x62b4df (0x9370 bytes) 000232[0] 1781046638.786509 mapwrite 0x132acd thru 0x1378e7 (0x4e1b bytes) 000233[0] 1781046638.803073 write 0x28885c thru 0x296c20 (0xe3c5 bytes) 000236[0] 1781046638.814243 mapread 0x206b2c thru 0x213a4e (0xcf23 bytes) 000238[1] 1781046638.818757 mapwrite 0x983e5e thru 0x98a6ba (0x685d bytes) 000241[0] 1781046638.858954 read 0x696d0e thru 0x69eba2 (0x7e95 bytes) 000242[1] 1781046638.866735 mapread 0x607e3a thru 0x61675d (0xe924 bytes) 000245[1] 1781046638.890812 mapread 0x7c74e7 thru 0x7d3da6 (0xc8c0 bytes) 000247[1] 1781046638.908327 trunc from 0x98a6bb to 0x34dccc (0x63c9ee bytes) 000248[0] 1781046638.919075 write 0x4b2d8a thru 0x4b76d4 (0x494b bytes) 000249[1] 1781046638.925775 mapread 0x3f01e1 thru 0x3f9df4 (0x9c14 bytes) 000251[1] 1781046638.958190 mapwrite 0x6f3c9b thru 0x6f8f9f (0x5305 bytes) 000252[1] 1781046638.986776 mapread 0x06c332 thru 0x07435f (0x802e bytes) 000255[0] 1781046639.008591 mapwrite 0x62b0a5 thru 0x631ab9 (0x6a15 bytes) 000259[0] 1781046639.045584 read 0x304634 thru 0x30c58c (0x7f59 bytes) 000261[1] 1781046639.078211 mapwrite 0x99acbc thru 0x9a35f6 (0x893b bytes) 000262[1] 1781046639.116480 read 0x2768ce thru 0x28274f (0xbe82 bytes) 000268[1] 1781046639.168925 write 0x31e86d thru 0x31f0b8 (0x84c bytes) 000269[1] 1781046639.172483 mapread 0x080f58 thru 0x08f1e6 (0xe28f bytes) 000270[1] 1781046639.191544 mapwrite 0x864131 thru 0x86e3b9 (0xa289 bytes) 000272[1] 1781046639.205205 read 0x8edd5b thru 0x8fa0d8 (0xc37e bytes) 000274[0] 1781046639.213156 mapread 0x074cc7 thru 0x07fc48 (0xaf82 bytes) 000275[1] 1781046639.235570 write 0x18560f thru 0x191ba0 (0xc592 bytes) 000277[0] 1781046639.241980 trunc from 0x9a35f7 to 0x1ddd39 (0x7c58bd bytes) 000280[1] 1781046639.267857 read 0x0a9adf thru 0x0ab362 (0x1884 bytes) 000281[0] 1781046639.274806 mapwrite 0x3e5e52 thru 0x3e961e (0x37cd bytes) 000284[1] 1781046639.312267 trunc from 0x3e961f to 0x5665b3 (0x17cf95 bytes) 000285[1] 1781046639.322956 mapread 0x232f90 thru 0x23d180 (0xa1f1 bytes) 000287[0] 1781046639.362462 mapread 0x28fe29 thru 0x29897e (0x8b56 bytes) 000288[1] 1781046639.382689 trunc from 0x5665b4 to 0x1b119e (0x3b5415 bytes) 000290[0] 1781046639.395412 trunc from 0x1b119f to 0x3cdaa7 (0x21c909 bytes) 000291[0] 1781046639.404615 mapwrite 0x2caf6b thru 0x2d3990 (0x8a26 bytes) 000293[1] 1781046639.438250 write 0x5f9c30 thru 0x601e85 (0x8256 bytes) 000296[1] 1781046639.447303 trunc from 0x601e86 to 0x90e471 (0x30c5ec bytes) 000297[0] 1781046639.452834 mapread 0x540f27 thru 0x548c58 (0x7d32 bytes) 000298[1] 1781046639.477863 trunc from 0x90e472 to 0x94c45a (0x3dfe9 bytes) 000299[1] 1781046639.484857 write 0x6264f9 thru 0x62857e (0x2086 bytes) 000300[0] 1781046639.491045 write 0x624cf3 thru 0x63339d (0xe6ab bytes) 000302[1] 1781046639.512165 write 0x816aeb thru 0x817640 (0xb56 bytes) 000304[1] 1781046639.528254 mapread 0x2de678 thru 0x2ec26b (0xdbf4 bytes) 000305[1] 1781046639.544795 mapread 0x53676b thru 0x53b167 (0x49fd bytes) 000306[1] 1781046639.555310 mapread 0x91cb30 thru 0x927b95 (0xb066 bytes) 000308[0] 1781046639.576331 write 0x21ea45 thru 0x22bab2 (0xd06e bytes) 000309[0] 1781046639.585811 mapwrite 0x0efb8f thru 0x0f6cd0 (0x7142 bytes) 000310[0] 1781046639.614927 write 0x07fba8 thru 0x084819 (0x4c72 bytes) 000312[1] 1781046639.624668 trunc from 0x94c45b to 0x4a78b1 (0x4a4ba9 bytes) 000314[0] 1781046639.640819 mapwrite 0x81f2d0 thru 0x82e121 (0xee52 bytes) 000316[0] 1781046639.686215 mapwrite 0x646357 thru 0x64f72a (0x93d4 bytes) 000321[0] 1781046639.716606 mapread 0x6cb40c thru 0x6daf8a (0xfb7f bytes) 000322[0] 1781046639.736397 read 0x1a7c84 thru 0x1b3c4f (0xbfcc bytes) 000323[1] 1781046639.777310 mapwrite 0x6cba07 thru 0x6cbdb9 (0x3b3 bytes) 000324[1] 1781046639.804627 mapwrite 0x5b7c5a thru 0x5bbc12 (0x3fb9 bytes) 000328[0] 1781046639.846395 mapwrite 0x5428b5 thru 0x54f2c5 (0xca11 bytes) 000329[0] 1781046639.889385 read 0x7136dc thru 0x715bf0 (0x2515 bytes) 000331[0] 1781046639.907458 read 0x240e78 thru 0x2468ef (0x5a78 bytes) 000332[0] 1781046639.948081 mapread 0x02000a thru 0x02a7fa (0xa7f1 bytes) 000333[1] 1781046639.970855 write 0x76f72c thru 0x7715d9 (0x1eae bytes) 000334[0] 1781046639.980760 mapwrite 0x5120f4 thru 0x51abe6 (0x8af3 bytes) 000335[1] 1781046640.021902 read 0x28ddac thru 0x29b3a5 (0xd5fa bytes) 000336[1] 1781046640.033140 read 0x5c7efe thru 0x5ca8f7 (0x29fa bytes) 000337[0] 1781046640.058638 mapwrite 0x9b4fcf thru 0x9bef70 (0x9fa2 bytes) 000339[0] 1781046640.118113 write 0x7da30b thru 0x7e4a97 (0xa78d bytes) 000342[1] 1781046640.133610 read 0x1e9191 thru 0x1f4f1e (0xbd8e bytes) 000343[0] 1781046640.146986 read 0x5f8065 thru 0x600e72 (0x8e0e bytes) 000345[1] 1781046640.180550 write 0x9af18f thru 0x9b0e6e (0x1ce0 bytes) 000347[0] 1781046640.197036 read 0x8d5939 thru 0x8d90c6 (0x378e bytes) 000348[0] 1781046640.201897 read 0x25feea thru 0x26db11 (0xdc28 bytes) 000350[1] 1781046640.217496 mapwrite 0x83a2ae thru 0x83e2ed (0x4040 bytes) 000352[1] 1781046640.302671 write 0x1ecc22 thru 0x1f3a7f (0x6e5e bytes) 000353[0] 1781046640.306241 read 0x6a4fb8 thru 0x6a545f (0x4a8 bytes) 000354[1] 1781046640.319336 read 0x14f17d thru 0x14f28d (0x111 bytes) 000355[1] 1781046640.344888 read 0x94e409 thru 0x95959e (0xb196 bytes) 000358[1] 1781046640.356046 trunc from 0x9bef71 to 0x0e6feb (0x8d7f85 bytes) 000360[0] 1781046640.377572 mapwrite 0x4694fe thru 0x474a86 (0xb589 bytes) 000362[1] 1781046640.417128 write 0x111a26 thru 0x12188d (0xfe68 bytes) 000363[1] 1781046640.425232 mapwrite 0x8d70ac thru 0x8dd29c (0x61f1 bytes) 000364[1] 1781046640.453470 trunc from 0x8dd29d to 0x59390c (0x349990 bytes) 000365[1] 1781046640.457414 trunc from 0x59390d to 0x886675 (0x2f2d69 bytes) 000369[1] 1781046640.473687 trunc from 0x886676 to 0x2f097c (0x595cf9 bytes) 000372[1] 1781046640.482798 read 0x16d424 thru 0x1781d9 (0xadb6 bytes) 000376[1] 1781046640.500373 mapread 0x19bb42 thru 0x1a1e89 (0x6348 bytes) 000377[0] 1781046640.500903 mapwrite 0x39d4da thru 0x3a64f4 (0x901b bytes) 000378[0] 1781046640.538589 mapread 0x052e18 thru 0x05b41e (0x8607 bytes) 000379[0] 1781046640.551023 trunc from 0x3a64f5 to 0x9eb867 (0x645373 bytes) 000380[1] 1781046640.559802 read 0x307801 thru 0x3109ab (0x91ab bytes) 000381[0] 1781046640.573086 read 0x0daade thru 0x0e65da (0xbafd bytes) 000382[1] 1781046640.578365 write 0x0773e7 thru 0x0776ad (0x2c7 bytes) 000383[0] 1781046640.615605 mapread 0x045d71 thru 0x048d0a (0x2f9a bytes) 000384[0] 1781046640.672208 write 0x78ba17 thru 0x78cc51 (0x123b bytes) 000385[0] 1781046640.686391 trunc from 0x9eb868 to 0x29f9bf (0x74bea8 bytes) 000387[0] 1781046640.702859 read 0x129c32 thru 0x136ac7 (0xce96 bytes) 000388[0] 1781046640.720053 write 0x1c5551 thru 0x1cf175 (0x9c25 bytes) 000389[0] 1781046640.724423 mapwrite 0x31f72a thru 0x323d66 (0x463d bytes) 000390[1] 1781046640.750201 trunc from 0x323d67 to 0x5b54e4 (0x29177e bytes) 000392[1] 1781046640.763060 read 0x4f51bb thru 0x5005dd (0xb423 bytes) 000399[0] 1781046640.786888 mapwrite 0x9d4829 thru 0x9e0f6b (0xc743 bytes) 000403[1] 1781046640.831072 read 0x8a1170 thru 0x8a2675 (0x1506 bytes) 000405[1] 1781046640.843640 read 0x8c08b8 thru 0x8cac0d (0xa356 bytes) 000406[1] 1781046640.848141 trunc from 0x9e0f6c to 0x097a93 (0x9494d8 bytes) 000407[0] 1781046640.861444 mapread 0x04394e thru 0x05353a (0xfbed bytes) 000408[1] 1781046640.886680 write 0x01b169 thru 0x0202e1 (0x5179 bytes) 000409[1] 1781046640.891847 write 0x53bf12 thru 0x5484e7 (0xc5d6 bytes) 000410[1] 1781046640.893565 trunc from 0x5484e8 to 0x842131 (0x2f9c4a bytes) 000413[0] 1781046640.907301 mapwrite 0x4d0f73 thru 0x4d8382 (0x7410 bytes) 000414[0] 1781046640.941797 mapread 0x3abe9b thru 0x3b6495 (0xa5fb bytes) 000417[1] 1781046640.967869 read 0x42d104 thru 0x43cab8 (0xf9b5 bytes) 000418[1] 1781046640.996899 read 0x0fcc55 thru 0x100d83 (0x412f bytes) 000419[0] 1781046641.022771 read 0x3b5137 thru 0x3bdacd (0x8997 bytes) 000420[0] 1781046641.052729 read 0x27c55d thru 0x28520f (0x8cb3 bytes) 000424[0] 1781046641.063643 mapwrite 0x7658be thru 0x76a0d4 (0x4817 bytes) 000427[0] 1781046641.108686 mapread 0x25a8ae thru 0x2699a6 (0xf0f9 bytes) 000429[1] 1781046641.130968 mapread 0x4059ff thru 0x40ac07 (0x5209 bytes) 000430[1] 1781046641.136086 read 0x67c5e0 thru 0x67f213 (0x2c34 bytes) 000432[0] 1781046641.136627 mapwrite 0x1f92ca thru 0x201883 (0x85ba bytes) 000435[0] 1781046641.177742 mapwrite 0x3e0d10 thru 0x3e25eb (0x18dc bytes) 000438[1] 1781046641.197787 read 0x796702 thru 0x79df6f (0x786e bytes) 000439[0] 1781046641.207238 mapread 0x718235 thru 0x71df31 (0x5cfd bytes) 000440[1] 1781046641.218516 write 0x5d0cf6 thru 0x5dee76 (0xe181 bytes) 000441[0] 1781046641.227953 write 0x1ca0ce thru 0x1cf90d (0x5840 bytes) 000447[1] 1781046641.253204 read 0x018345 thru 0x0265d1 (0xe28d bytes) 000449[1] 1781046641.269699 write 0x956364 thru 0x95d384 (0x7021 bytes) 000452[0] 1781046641.285015 mapwrite 0x5e5033 thru 0x5ea98b (0x5959 bytes) 000456[0] 1781046641.321660 trunc from 0x95d385 to 0x15a5b7 (0x802dcd bytes) 000457[1] 1781046641.331119 trunc from 0x15a5b8 to 0x4717d6 (0x31721f bytes) 000458[0] 1781046641.340512 mapwrite 0x7902e3 thru 0x7936cb (0x33e9 bytes) 000460[1] 1781046641.372717 write 0x1a69e9 thru 0x1ad6ad (0x6cc5 bytes) 000462[1] 1781046641.384676 mapread 0x766ef7 thru 0x76cfea (0x60f4 bytes) 000463[0] 1781046641.394586 trunc from 0x7936cc to 0x08b528 (0x7081a3 bytes) 000465[1] 1781046641.411645 mapwrite 0x537d04 thru 0x540ce9 (0x8fe6 bytes) 000467[0] 1781046641.442793 trunc from 0x540cea to 0x274608 (0x2cc6e1 bytes) 000470[0] 1781046641.460500 trunc from 0x274609 to 0x46536d (0x1f0d65 bytes) 000472[1] 1781046641.470298 mapwrite 0x572336 thru 0x57e2d4 (0xbf9f bytes) 000474[1] 1781046641.505973 trunc from 0x57e2d5 to 0x8dbcb8 (0x35d9e4 bytes) 000475[0] 1781046641.515606 mapread 0x6ec215 thru 0x6f1fba (0x5da6 bytes) 000476[1] 1781046641.527241 write 0x8b1c2f thru 0x8c084c (0xec1e bytes) 000477[0] 1781046641.532021 trunc from 0x8dbcb9 to 0x5750be (0x366bfa bytes) 000479[1] 1781046641.546648 mapwrite 0x6e99e6 thru 0x6f9715 (0xfd30 bytes) 000480[1] 1781046641.576960 mapread 0x4b3813 thru 0x4c2e93 (0xf681 bytes) 000485[1] 1781046641.593248 write 0x39efc2 thru 0x3a842c (0x946b bytes) 000488[0] 1781046641.602858 trunc from 0x6f9716 to 0x49959d (0x260178 bytes) 000489[1] 1781046641.613216 read 0x05348b thru 0x0570a7 (0x3c1d bytes) 000491[0] 1781046641.622803 read 0x373ee9 thru 0x376bab (0x2cc3 bytes) 000492[0] 1781046641.632493 mapwrite 0x860c2e thru 0x86e964 (0xdd37 bytes) 000494[0] 1781046641.684630 trunc from 0x86e965 to 0x58881e (0x2e6146 bytes) 000496[1] 1781046641.696880 trunc from 0x58881f to 0x275da3 (0x312a7b bytes) 000500[1] 1781046641.715551 write 0x429883 thru 0x431563 (0x7ce1 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 6641 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[1] 1781046641.931400 mapwrite 0x0a6000 thru 0x0aa2b4 (0x42b5 bytes) truncating to largest ever: 0x72ceb4 000005[0] 1781046641.972957 trunc from 0x0aa2b5 to 0x72ceb3 (0x682bff bytes) 000006[1] 1781046641.980739 write_OD 0x234000 thru 0x240fff (0xd000 bytes) 000007[0] 1781046641.995272 mapwrite 0x9e0000 thru 0x9eb485 (0xb486 bytes) 000008[1] 1781046642.023601 mapread 0x279000 thru 0x27f4af (0x64b0 bytes) 000011[1] 1781046642.034887 read_OD 0x4cb000 thru 0x4d7fff (0xd000 bytes) 000012[0] 1781046642.037232 write 0x5fb000 thru 0x604e21 (0x9e22 bytes) 000014[1] 1781046642.044393 trunc from 0x9eb486 to 0x581c16 (0x46986f bytes) 000015[0] 1781046642.052655 write 0x1d3000 thru 0x1dd363 (0xa364 bytes) 000016[0] 1781046642.059385 trunc from 0x581c17 to 0x6568c4 (0xd4cae bytes) 000018[1] 1781046642.063977 write_OD 0x201000 thru 0x20efff (0xe000 bytes) truncating to largest ever: 0x918237 000020[1] 1781046642.082960 trunc from 0x6568c5 to 0x918236 (0x2c1972 bytes) 000023[1] 1781046642.095489 mapread 0x1de000 thru 0x1eac2b (0xcc2c bytes) 000029[0] 1781046642.115973 read 0x190000 thru 0x194f82 (0x4f83 bytes) 000030[0] 1781046642.121434 mapwrite 0x2cc000 thru 0x2d833b (0xc33c bytes) 000032[0] 1781046642.147197 mapwrite 0x7d1000 thru 0x7d396e (0x296f bytes) 000033[1] 1781046642.167031 mapwrite 0x71e000 thru 0x7275c9 (0x95ca bytes) 000034[0] 1781046642.187449 mapread 0x71b000 thru 0x71f3f5 (0x43f6 bytes) 000036[1] 1781046642.194227 trunc from 0x918237 to 0x42ff5f (0x4e82d7 bytes) 000042[0] 1781046642.212749 read 0x2a4000 thru 0x2ae483 (0xa484 bytes) 000044[0] 1781046642.227931 mapwrite 0x2e5000 thru 0x2f28df (0xd8e0 bytes) 000046[0] 1781046642.276479 mapread 0x1d0000 thru 0x1d1229 (0x122a bytes) 000047[1] 1781046642.278871 trunc from 0x42ff60 to 0x2ca329 (0x165c36 bytes) 000048[0] 1781046642.288260 mapread 0x2a2000 thru 0x2a9710 (0x7711 bytes) 000051[0] 1781046642.303796 mapread 0x192000 thru 0x19b729 (0x972a bytes) 000054[0] 1781046642.319384 read 0x039000 thru 0x03ccb8 (0x3cb9 bytes) 000056[1] 1781046642.338328 trunc from 0x2ca32a to 0x68edfa (0x3c4ad1 bytes) 000058[0] 1781046642.350079 write 0x0b9000 thru 0x0bee41 (0x5e42 bytes) 000060[1] 1781046642.385811 trunc from 0x68edfb to 0x30cf9a (0x381e60 bytes) 000062[1] 1781046642.401238 mapwrite 0x9bc000 thru 0x9c8b23 (0xcb24 bytes) 000063[0] 1781046642.429677 mapwrite 0x11c000 thru 0x124863 (0x8864 bytes) 000064[0] 1781046642.488711 mapwrite 0x28c000 thru 0x293bcd (0x7bce bytes) 000066[0] 1781046642.518085 write 0x6a7000 thru 0x6a8489 (0x148a bytes) 000072[1] 1781046642.532538 mapread 0x179000 thru 0x181444 (0x8445 bytes) 000073[1] 1781046642.547149 write_OD 0x310000 thru 0x315fff (0x6000 bytes) 000074[1] 1781046642.561165 write_OD 0x0db000 thru 0x0e1fff (0x7000 bytes) 000075[1] 1781046642.574289 write_OD 0x58e000 thru 0x58efff (0x1000 bytes) 000078[1] 1781046642.586631 trunc from 0x9c8b24 to 0x01671e (0x9b2405 bytes) 000080[1] 1781046642.600211 read_OD 0x007000 thru 0x007fff (0x1000 bytes) 000081[1] 1781046642.604605 mapwrite 0x75e000 thru 0x7640f8 (0x60f9 bytes) 000083[1] 1781046642.672665 mapwrite 0x891000 thru 0x891ee4 (0xee5 bytes) 000084[1] 1781046642.691563 write_OD 0x4bb000 thru 0x4c7fff (0xd000 bytes) 000085[0] 1781046642.711982 read 0x2ea000 thru 0x2f7a39 (0xda3a bytes) 000086[0] 1781046642.746460 read 0x114000 thru 0x1235fd (0xf5fe bytes) 000087[0] 1781046642.761949 trunc from 0x891ee5 to 0x1a568d (0x6ec857 bytes) 000088[0] 1781046642.776936 trunc from 0x1a568e to 0x7337d1 (0x58e144 bytes) 000090[1] 1781046642.781358 mapread 0x680000 thru 0x68817d (0x817e bytes) 000091[0] 1781046642.795752 mapwrite 0x2e1000 thru 0x2eb8f9 (0xa8fa bytes) 000092[1] 1781046642.833607 mapread 0x5fe000 thru 0x60b8c7 (0xd8c8 bytes) 000093[0] 1781046642.854617 mapwrite 0x548000 thru 0x54c160 (0x4161 bytes) 000097[1] 1781046642.878869 write_OD 0x2ff000 thru 0x304fff (0x6000 bytes) 000099[1] 1781046642.892067 mapread 0x450000 thru 0x4560c9 (0x60ca bytes) 000100[1] 1781046642.899346 mapwrite 0x056000 thru 0x05a9c2 (0x49c3 bytes) 000102[1] 1781046642.921914 trunc from 0x7337d2 to 0x88f022 (0x15b851 bytes) 000104[0] 1781046642.932893 trunc from 0x88f023 to 0x879164 (0x15ebe bytes) 000105[1] 1781046642.938060 write_OD 0x81b000 thru 0x825fff (0xb000 bytes) 000106[0] 1781046642.952596 mapread 0x2fe000 thru 0x3094af (0xb4b0 bytes) 000107[1] 1781046642.967601 trunc from 0x879165 to 0x06032d (0x818e37 bytes) 000109[0] 1781046642.982599 mapread 0x017000 thru 0x017daf (0xdb0 bytes) 000110[1] 1781046642.988851 trunc from 0x06032e to 0x89df6d (0x83dc40 bytes) 000112[1] 1781046643.005209 trunc from 0x89df6e to 0x2c6b9c (0x5d73d1 bytes) 000113[0] 1781046643.013242 read 0x04e000 thru 0x05dca4 (0xfca5 bytes) 000118[0] 1781046643.037297 read 0x03e000 thru 0x03e4ff (0x500 bytes) 000119[1] 1781046643.040312 mapwrite 0x1dd000 thru 0x1ebb7f (0xeb80 bytes) 000121[0] 1781046643.077324 trunc from 0x2c6b9d to 0x2281fc (0x9e9a0 bytes) 000125[0] 1781046643.091175 mapwrite 0x8d4000 thru 0x8dc83c (0x883d bytes) 000127[1] 1781046643.147381 read_OD 0x15d000 thru 0x166fff (0xa000 bytes) 000129[1] 1781046643.159955 trunc from 0x8dc83d to 0x59a57d (0x3422bf bytes) 000131[0] 1781046643.171053 write 0x025000 thru 0x02c841 (0x7842 bytes) 000132[1] 1781046643.174879 write_OD 0x921000 thru 0x92ffff (0xf000 bytes) 000133[0] 1781046643.191600 read 0x0b1000 thru 0x0b2c8f (0x1c90 bytes) 000134[0] 1781046643.241791 mapread 0x638000 thru 0x640837 (0x8838 bytes) 000136[0] 1781046643.244187 trunc from 0x930000 to 0x4a20a0 (0x48df5f bytes) 000138[1] 1781046643.258628 trunc from 0x4a20a1 to 0x8c7cec (0x425c4c bytes) 000142[0] 1781046643.269966 mapwrite 0x843000 thru 0x850c1e (0xdc1f bytes) 000144[0] 1781046643.305578 trunc from 0x8c7ced to 0x191719 (0x7365d3 bytes) 000145[1] 1781046643.310157 trunc from 0x19171a to 0x0c2ad5 (0xcec44 bytes) 000146[1] 1781046643.319258 read_OD 0x05b000 thru 0x05bfff (0x1000 bytes) 000147[1] 1781046643.322641 write_OD 0x3a2000 thru 0x3abfff (0xa000 bytes) 000149[0] 1781046643.339398 read 0x188000 thru 0x188a2f (0xa30 bytes) 000156[1] 1781046643.357691 mapread 0x3a9000 thru 0x3ab73c (0x273d bytes) 000157[0] 1781046643.382892 trunc from 0x3ac000 to 0x85012a (0x4a412b bytes) 000161[0] 1781046643.407164 read 0x2d0000 thru 0x2d3c66 (0x3c67 bytes) 000162[1] 1781046643.438667 trunc from 0x85012b to 0x3bc6ae (0x493a7c bytes) 000163[0] 1781046643.454521 mapwrite 0x0be000 thru 0x0beb45 (0xb46 bytes) 000164[0] 1781046643.474510 write 0x4c5000 thru 0x4c8d11 (0x3d12 bytes) 000166[1] 1781046643.478446 mapread 0x358000 thru 0x364ced (0xccee bytes) 000167[0] 1781046643.498508 mapread 0x004000 thru 0x013ed6 (0xfed7 bytes) 000168[0] 1781046643.516024 write 0x8a4000 thru 0x8a918f (0x5190 bytes) 000170[0] 1781046643.523442 write 0x367000 thru 0x372bbe (0xbbbf bytes) 000172[0] 1781046643.530260 trunc from 0x8a9190 to 0x44be9a (0x45d2f5 bytes) 000174[0] 1781046643.541572 write 0x99c000 thru 0x9a191c (0x591d bytes) 000176[0] 1781046643.547885 trunc from 0x9a191d to 0x4a7f7b (0x4f99a1 bytes) 000178[0] 1781046643.556615 write 0x56b000 thru 0x57ab12 (0xfb13 bytes) 000180[0] 1781046643.560938 mapread 0x090000 thru 0x0986a4 (0x86a5 bytes) 000181[0] 1781046643.570435 mapwrite 0x2dc000 thru 0x2ea728 (0xe729 bytes) 000183[0] 1781046643.604957 write 0x2d7000 thru 0x2e3fa0 (0xcfa1 bytes) 000184[1] 1781046643.605625 read_OD 0x34e000 thru 0x356fff (0x9000 bytes) 000185[0] 1781046643.613452 read 0x022000 thru 0x022fbf (0xfc0 bytes) 000187[1] 1781046643.651038 write_OD 0x009000 thru 0x011fff (0x9000 bytes) 000188[1] 1781046643.668059 read_OD 0x2e2000 thru 0x2e2fff (0x1000 bytes) 000189[1] 1781046643.670407 mapread 0x0fa000 thru 0x0fbc21 (0x1c22 bytes) 000190[1] 1781046643.689116 trunc from 0x57ab13 to 0x879718 (0x2fec06 bytes) 000191[0] 1781046643.702504 mapread 0x034000 thru 0x03e80e (0xa80f bytes) 000193[1] 1781046643.744949 mapwrite 0x617000 thru 0x6229dd (0xb9de bytes) 000194[0] 1781046643.793743 read 0x5da000 thru 0x5e27e5 (0x87e6 bytes) 000195[1] 1781046643.816043 write_OD 0x670000 thru 0x674fff (0x5000 bytes) 000196[1] 1781046643.830538 write_OD 0x868000 thru 0x86cfff (0x5000 bytes) 000197[0] 1781046643.842232 write 0x135000 thru 0x13922b (0x422c bytes) 000200[1] 1781046643.852274 trunc from 0x879719 to 0x479e34 (0x3ff8e4 bytes) 000201[1] 1781046643.863322 read_OD 0x079000 thru 0x07efff (0x6000 bytes) 000202[1] 1781046643.868014 mapread 0x1fd000 thru 0x20c103 (0xf104 bytes) 000203[1] 1781046643.894228 mapwrite 0x257000 thru 0x260191 (0x9192 bytes) 000204[1] 1781046643.937570 mapread 0x3e5000 thru 0x3e726b (0x226c bytes) 000205[0] 1781046643.944804 mapwrite 0x56a000 thru 0x572a32 (0x8a33 bytes) 000208[1] 1781046643.978558 write_OD 0x82d000 thru 0x835fff (0x9000 bytes) 000209[0] 1781046643.991445 read 0x7a3000 thru 0x7a7c7b (0x4c7c bytes) 000210[1] 1781046643.997853 write_OD 0x7e9000 thru 0x7f6fff (0xe000 bytes) 000213[1] 1781046644.012179 read_OD 0x037000 thru 0x03cfff (0x6000 bytes) 000216[1] 1781046644.014641 trunc from 0x836000 to 0x8e7281 (0xb1282 bytes) 000217[0] 1781046644.024480 read 0x744000 thru 0x74eab8 (0xaab9 bytes) 000218[1] 1781046644.040316 write_OD 0x0ef000 thru 0x0fdfff (0xf000 bytes) 000219[0] 1781046644.057420 read 0x793000 thru 0x7a0088 (0xd089 bytes) 000220[1] 1781046644.068709 trunc from 0x8e7282 to 0x59ade5 (0x34c49c bytes) 000221[1] 1781046644.080247 write_OD 0x470000 thru 0x47bfff (0xc000 bytes) 000222[1] 1781046644.094324 trunc from 0x59ade6 to 0x4ff5a7 (0x9b83e bytes) 000224[0] 1781046644.105051 mapread 0x2e2000 thru 0x2f078b (0xe78c bytes) 000225[0] 1781046644.124176 mapwrite 0x2a9000 thru 0x2b4356 (0xb357 bytes) 000227[0] 1781046644.153998 mapwrite 0x14f000 thru 0x1595d2 (0xa5d3 bytes) 000230[0] 1781046644.187710 read 0x329000 thru 0x336819 (0xd81a bytes) 000231[1] 1781046644.201632 mapread 0x316000 thru 0x31da1b (0x7a1c bytes) 000232[0] 1781046644.239346 mapread 0x451000 thru 0x45a826 (0x9827 bytes) 000233[0] 1781046644.250540 mapwrite 0x085000 thru 0x08fa18 (0xaa19 bytes) 000237[1] 1781046644.286722 write_OD 0x581000 thru 0x58ffff (0xf000 bytes) 000244[0] 1781046644.300537 read 0x07f000 thru 0x080dec (0x1ded bytes) 000245[0] 1781046644.329642 trunc from 0x590000 to 0x8d2a15 (0x342a16 bytes) 000247[1] 1781046644.348996 read_OD 0x4fa000 thru 0x501fff (0x8000 bytes) 000248[0] 1781046644.355074 mapread 0x612000 thru 0x61733c (0x533d bytes) 000249[1] 1781046644.380504 mapwrite 0x714000 thru 0x721982 (0xd983 bytes) 000251[1] 1781046644.429250 read_OD 0x784000 thru 0x787fff (0x4000 bytes) 000255[1] 1781046644.437705 read_OD 0x56d000 thru 0x578fff (0xc000 bytes) 000259[1] 1781046644.440082 mapwrite 0x9de000 thru 0x9e5081 (0x7082 bytes) 000260[1] 1781046644.493160 write_OD 0x20b000 thru 0x211fff (0x7000 bytes) 000261[0] 1781046644.511909 read 0x66e000 thru 0x67da2b (0xfa2c bytes) 000265[0] 1781046644.534655 read 0x04e000 thru 0x055e64 (0x7e65 bytes) 000266[0] 1781046644.560047 trunc from 0x9e5082 to 0x68adef (0x35a292 bytes) 000267[0] 1781046644.583309 trunc from 0x68adf0 to 0x13ccc0 (0x54e12f bytes) 000269[0] 1781046644.588526 mapread 0x12c000 thru 0x13143d (0x543e bytes) 000270[1] 1781046644.595342 mapread 0x0ca000 thru 0x0d0e9d (0x6e9e bytes) 000271[0] 1781046644.607082 read 0x100000 thru 0x10e55d (0xe55e bytes) 000272[0] 1781046644.611327 mapread 0x107000 thru 0x10a330 (0x3331 bytes) 000273[0] 1781046644.611608 trunc from 0x13ccc1 to 0x8bbd72 (0x77f0b2 bytes) 000274[0] 1781046644.622981 read 0x09d000 thru 0x09f6a4 (0x26a5 bytes) 000276[1] 1781046644.668313 read_OD 0x1ba000 thru 0x1bdfff (0x4000 bytes) 000277[0] 1781046644.680730 mapread 0x37e000 thru 0x37fca5 (0x1ca6 bytes) 000279[0] 1781046644.684415 read 0x55b000 thru 0x560852 (0x5853 bytes) 000283[1] 1781046644.709216 trunc from 0x8bbd73 to 0x05f2ff (0x85ca73 bytes) 000284[1] 1781046644.721586 read_OD 0x008000 thru 0x014fff (0xd000 bytes) 000285[1] 1781046644.726369 read_OD 0x04c000 thru 0x04cfff (0x1000 bytes) 000286[1] 1781046644.727631 mapwrite 0x797000 thru 0x7a3df1 (0xcdf2 bytes) 000288[0] 1781046644.796612 mapread 0x00b000 thru 0x012982 (0x7983 bytes) 000292[0] 1781046644.816441 read 0x3e0000 thru 0x3ea3b0 (0xa3b1 bytes) 000294[1] 1781046644.837749 read_OD 0x57f000 thru 0x583fff (0x5000 bytes) 000296[1] 1781046644.840284 read_OD 0x5b0000 thru 0x5b8fff (0x9000 bytes) 000297[0] 1781046644.842308 mapread 0x3fe000 thru 0x406ef4 (0x8ef5 bytes) 000299[0] 1781046644.842895 mapread 0x00c000 thru 0x0193c9 (0xd3ca bytes) 000300[0] 1781046644.851540 read 0x755000 thru 0x760e19 (0xbe1a bytes) 000301[0] 1781046644.851754 trunc from 0x7a3df2 to 0x243211 (0x560be0 bytes) 000302[1] 1781046644.866048 write_OD 0x6c7000 thru 0x6d5fff (0xf000 bytes) 000304[1] 1781046644.880475 write_OD 0x11e000 thru 0x123fff (0x6000 bytes) 000305[1] 1781046644.892743 mapwrite 0x0f3000 thru 0x0fe20d (0xb20e bytes) 000306[0] 1781046644.930171 write 0x9b9000 thru 0x9ba55e (0x155f bytes) 000307[0] 1781046644.934628 mapwrite 0x09c000 thru 0x09cfd2 (0xfd3 bytes) truncating to largest ever: 0x992e25 000308[1] 1781046644.966266 trunc from 0x9ba55f to 0x992e24 (0x2773a bytes) 000310[1] 1781046644.978653 write_OD 0x886000 thru 0x890fff (0xb000 bytes) 000312[1] 1781046644.990657 mapread 0x791000 thru 0x79b1cc (0xa1cd bytes) 000314[1] 1781046645.010718 read_OD 0x95d000 thru 0x964fff (0x8000 bytes) 000316[0] 1781046645.013045 mapwrite 0x827000 thru 0x8336ed (0xc6ee bytes) 000318[1] 1781046645.046540 mapwrite 0x959000 thru 0x96645a (0xd45b bytes) truncating to largest ever: 0x9a326a 000322[1] 1781046645.080357 trunc from 0x992e25 to 0x9a3269 (0x10445 bytes) 000323[0] 1781046645.084769 read 0x0d1000 thru 0x0e0541 (0xf542 bytes) 000324[0] 1781046645.131333 read 0x7a4000 thru 0x7ae1e6 (0xa1e7 bytes) 000326[0] 1781046645.136035 mapwrite 0x952000 thru 0x95c9dd (0xa9de bytes) 000327[0] 1781046645.174683 read 0x029000 thru 0x02c025 (0x3026 bytes) 000330[0] 1781046645.223158 trunc from 0x9a326a to 0x57005b (0x43320e bytes) 000331[1] 1781046645.234371 write_OD 0x4a8000 thru 0x4a8fff (0x1000 bytes) 000332[0] 1781046645.253676 read 0x4b8000 thru 0x4c4431 (0xc432 bytes) 000333[0] 1781046645.260715 trunc from 0x57005c to 0x0bf4f4 (0x4b0b67 bytes) 000336[0] 1781046645.278395 trunc from 0x0bf4f5 to 0x918315 (0x858e21 bytes) 000337[0] 1781046645.287604 mapread 0x068000 thru 0x0740d6 (0xc0d7 bytes) 000338[1] 1781046645.322993 mapwrite 0x4d6000 thru 0x4d957a (0x357b bytes) 000339[0] 1781046645.349668 write 0x410000 thru 0x410738 (0x739 bytes) 000341[0] 1781046645.356724 trunc from 0x918316 to 0x4df72d (0x438be8 bytes) 000345[0] 1781046645.371777 trunc from 0x4df72e to 0x4130da (0xcc653 bytes) 000348[0] 1781046645.380533 write 0x9e3000 thru 0x9efe1e (0xce1f bytes) 000349[1] 1781046645.382771 read_OD 0x6c1000 thru 0x6c5fff (0x5000 bytes) 000350[0] 1781046645.390417 mapwrite 0x175000 thru 0x1827db (0xd7dc bytes) 000351[1] 1781046645.429594 read_OD 0x44b000 thru 0x452fff (0x8000 bytes) 000352[1] 1781046645.437048 mapread 0x208000 thru 0x211de7 (0x9de8 bytes) 000354[0] 1781046645.482155 mapread 0x84f000 thru 0x85560f (0x6610 bytes) 000356[1] 1781046645.490647 write_OD 0x225000 thru 0x227fff (0x3000 bytes) 000357[1] 1781046645.508410 trunc from 0x9efe1f to 0x1f2904 (0x7fd51a bytes) 000359[1] 1781046645.515477 write_OD 0x7c1000 thru 0x7cdfff (0xd000 bytes) 000361[1] 1781046645.531478 trunc from 0x7ce000 to 0x064674 (0x76998b bytes) 000363[1] 1781046645.544208 trunc from 0x064675 to 0x02e640 (0x36034 bytes) 000364[0] 1781046645.548822 write 0x840000 thru 0x8493e6 (0x93e7 bytes) 000365[0] 1781046645.553981 read 0x739000 thru 0x746cbe (0xdcbf bytes) 000366[0] 1781046645.563503 mapread 0x520000 thru 0x5277e1 (0x77e2 bytes) 000370[1] 1781046645.578812 trunc from 0x8493e7 to 0x35a3e9 (0x4eeffd bytes) 000371[1] 1781046645.588286 mapwrite 0x597000 thru 0x599420 (0x2421 bytes) 000372[1] 1781046645.611397 mapread 0x4d2000 thru 0x4e1eeb (0xfeec bytes) 000375[1] 1781046645.631364 write_OD 0x9e6000 thru 0x9edfff (0x8000 bytes) 000376[1] 1781046645.645763 mapwrite 0x1b2000 thru 0x1b7f91 (0x5f92 bytes) 000377[0] 1781046645.669928 mapread 0x50a000 thru 0x5116f5 (0x76f6 bytes) 000379[1] 1781046645.695535 read_OD 0x76f000 thru 0x770fff (0x2000 bytes) 000380[1] 1781046645.696952 trunc from 0x9ee000 to 0x2a0eb7 (0x74d148 bytes) 000381[0] 1781046645.709243 trunc from 0x2a0eb8 to 0x6c39e8 (0x422b31 bytes) 000383[1] 1781046645.715349 trunc from 0x6c39e9 to 0x496073 (0x22d975 bytes) 000384[0] 1781046645.722025 mapread 0x458000 thru 0x463b1d (0xbb1e bytes) 000387[1] 1781046645.746031 mapwrite 0x18b000 thru 0x191673 (0x6674 bytes) 000390[0] 1781046645.774732 write 0x6dd000 thru 0x6e658c (0x958d bytes) 000392[0] 1781046645.781247 write 0x1e2000 thru 0x1ed1fd (0xb1fe bytes) 000393[0] 1781046645.783491 write 0x303000 thru 0x30ca60 (0x9a61 bytes) 000394[1] 1781046645.788583 read_OD 0x015000 thru 0x01dfff (0x9000 bytes) 000396[0] 1781046645.799801 trunc from 0x6e658d to 0x686dc9 (0x5f7c3 bytes) 000398[1] 1781046645.813999 mapwrite 0x4ce000 thru 0x4d06e1 (0x26e2 bytes) 000400[0] 1781046645.889679 trunc from 0x686dca to 0x401f3e (0x284e8b bytes) 000401[0] 1781046645.901827 mapread 0x2fb000 thru 0x2fda3d (0x2a3e bytes) 000402[1] 1781046645.909718 trunc from 0x401f3f to 0x5b9bcd (0x1b7c8f bytes) 000405[0] 1781046645.927080 mapread 0x379000 thru 0x380f13 (0x7f14 bytes) 000406[1] 1781046645.938596 trunc from 0x5b9bce to 0x5aa6d9 (0xf4f4 bytes) 000407[0] 1781046645.944608 trunc from 0x5aa6da to 0x3cc36d (0x1de36c bytes) 000413[0] 1781046645.963045 write 0x09c000 thru 0x0a57aa (0x97ab bytes) 000414[1] 1781046645.967945 read_OD 0x1a1000 thru 0x1a9fff (0x9000 bytes) 000415[0] 1781046645.975506 trunc from 0x3cc36e to 0x1a8c69 (0x223704 bytes) 000416[1] 1781046645.981914 read_OD 0x025000 thru 0x028fff (0x4000 bytes) 000419[1] 1781046645.989965 read_OD 0x0db000 thru 0x0dcfff (0x2000 bytes) 000420[0] 1781046645.991307 trunc from 0x1a8c6a to 0x05367b (0x1555ee bytes) 000421[0] 1781046646.000267 mapwrite 0x6b4000 thru 0x6bf78b (0xb78c bytes) 000424[0] 1781046646.038086 mapwrite 0x88a000 thru 0x88fe9a (0x5e9b bytes) 000425[1] 1781046646.069426 read_OD 0x5b4000 thru 0x5c1fff (0xe000 bytes) 000427[1] 1781046646.081511 mapread 0x698000 thru 0x6986c6 (0x6c7 bytes) 000428[0] 1781046646.110602 read 0x33b000 thru 0x340577 (0x5578 bytes) 000431[1] 1781046646.158821 mapread 0x580000 thru 0x58a9fe (0xa9ff bytes) 000432[1] 1781046646.175204 read_OD 0x860000 thru 0x860fff (0x1000 bytes) 000433[1] 1781046646.176971 mapwrite 0x67b000 thru 0x682989 (0x798a bytes) 000434[0] 1781046646.232032 mapread 0x522000 thru 0x522a37 (0xa38 bytes) 000436[1] 1781046646.238387 write_OD 0x85a000 thru 0x863fff (0xa000 bytes) 000437[1] 1781046646.250195 read_OD 0x0e1000 thru 0x0ecfff (0xc000 bytes) 000438[1] 1781046646.253383 trunc from 0x88fe9b to 0x5e1da4 (0x2ae0f6 bytes) 000440[0] 1781046646.265916 mapread 0x4b3000 thru 0x4bf39f (0xc3a0 bytes) 000442[0] 1781046646.285181 read 0x45d000 thru 0x464720 (0x7721 bytes) 000444[0] 1781046646.295666 read 0x191000 thru 0x19e452 (0xd453 bytes) 000445[1] 1781046646.317375 read_OD 0x0ab000 thru 0x0b7fff (0xd000 bytes) 000447[0] 1781046646.319970 mapread 0x54c000 thru 0x554dd8 (0x8dd9 bytes) 000448[0] 1781046646.320427 mapread 0x381000 thru 0x38cb33 (0xbb34 bytes) 000452[0] 1781046646.321045 trunc from 0x5e1da5 to 0x40159f (0x1e0805 bytes) 000455[1] 1781046646.339434 trunc from 0x4015a0 to 0x5a9464 (0x1a7ec5 bytes) 000457[1] 1781046646.349744 trunc from 0x5a9465 to 0x6af411 (0x105fad bytes) 000458[0] 1781046646.353839 read 0x1f4000 thru 0x203318 (0xf319 bytes) 000459[1] 1781046646.380481 write_OD 0x0f6000 thru 0x0fcfff (0x7000 bytes) 000460[1] 1781046646.399981 write_OD 0x33c000 thru 0x33ffff (0x4000 bytes) 000461[1] 1781046646.411144 mapread 0x180000 thru 0x18bd24 (0xbd25 bytes) 000463[0] 1781046646.445540 mapwrite 0x3ca000 thru 0x3d9f57 (0xff58 bytes) 000464[0] 1781046646.491143 trunc from 0x6af412 to 0x5ab2a4 (0x10416d bytes) 000467[1] 1781046646.495658 read_OD 0x0ea000 thru 0x0edfff (0x4000 bytes) 000468[1] 1781046646.501449 mapread 0x31c000 thru 0x3247b3 (0x87b4 bytes) 000471[1] 1781046646.532996 trunc from 0x5ab2a5 to 0x0e1bb8 (0x4c96ec bytes) 000476[1] 1781046646.552157 trunc from 0x0e1bb9 to 0x51f442 (0x43d88a bytes) 000477[1] 1781046646.560059 write_OD 0x7ec000 thru 0x7f4fff (0x9000 bytes) 000478[0] 1781046646.573511 mapread 0x4a5000 thru 0x4a7cf8 (0x2cf9 bytes) 000479[1] 1781046646.586861 read_OD 0x00f000 thru 0x00ffff (0x1000 bytes) 000480[1] 1781046646.588279 trunc from 0x7f5000 to 0x8244aa (0x2f4ab bytes) 000481[0] 1781046646.598956 trunc from 0x8244ab to 0x94ffbc (0x12bb12 bytes) 000483[0] 1781046646.609671 mapwrite 0x88c000 thru 0x88ce62 (0xe63 bytes) 000484[1] 1781046646.635180 read_OD 0x3a3000 thru 0x3aefff (0xc000 bytes) 000485[0] 1781046646.642390 mapread 0x35c000 thru 0x3656f4 (0x96f5 bytes) 000486[1] 1781046646.653156 read_OD 0x889000 thru 0x893fff (0xb000 bytes) 000487[1] 1781046646.654972 mapread 0x600000 thru 0x605240 (0x5241 bytes) 000488[0] 1781046646.666574 read 0x75d000 thru 0x768ff0 (0xbff1 bytes) 000489[0] 1781046646.681619 write 0x685000 thru 0x688392 (0x3393 bytes) 000491[1] 1781046646.689388 mapread 0x11c000 thru 0x124576 (0x8577 bytes) 000492[1] 1781046646.703081 write_OD 0x62e000 thru 0x62ffff (0x2000 bytes) 000496[0] 1781046646.715490 mapwrite 0x6ff000 thru 0x704da1 (0x5da2 bytes) 000497[0] 1781046646.735510 trunc from 0x94ffbd to 0x3a38c3 (0x5ac6f9 bytes) 000500[1] 1781046646.747298 write_OD 0x545000 thru 0x549fff (0x5000 bytes) All operations completed A-OK! PASS 16a (14s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 19:10:48 (1781046648) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0833432 s, 25.2 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0556151 s, 37.7 MB/s Chance of close/open is 1 in 50 Seed set to 6649 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x7f9c4 000001[0] 1781046649.239688 trunc from 00000000 to 0x07f9c3 (0x7f9c4 bytes) 000002[0] 1781046649.243328 mapwrite 0x682fe4 thru 0x68499a (0x19b7 bytes) 000003[1] 1781046649.278650 write 0x0eb278 thru 0x0ebf5c (0xce5 bytes) 000004[1] 1781046649.285465 mapwrite 0x125401 thru 0x125f09 (0xb09 bytes) 000005[1] 1781046649.302518 write 0x328cb0 thru 0x329fb5 (0x1306 bytes) 000007[1] 1781046649.314635 write 0x3a0033 thru 0x3a1f08 (0x1ed6 bytes) 000008[0] 1781046649.318421 write 0x90d525 thru 0x90e44c (0xf28 bytes) 000009[0] 1781046649.326450 mapread 0x49039e thru 0x4912f0 (0xf53 bytes) 000010[0] 1781046649.336861 mapread 0x540e49 thru 0x541f0b (0x10c3 bytes) truncating to largest ever: 0x7b3c34 000013[0] 1781046649.342828 trunc from 0x90e44d to 0x7b3c33 (0x15a819 bytes) 000014[0] 1781046649.350074 read 0x1d2333 thru 0x1d33ad (0x107b bytes) 000018[1] 1781046649.356649 read 0x0fc508 thru 0x0fe3cb (0x1ec4 bytes) 000021[1] 1781046649.367491 write 0x0451d9 thru 0x046286 (0x10ae bytes) 000022[0] 1781046649.375358 mapread 0x62e0db thru 0x62f37e (0x12a4 bytes) 000023[0] 1781046649.382507 read 0x28e961 thru 0x290072 (0x1712 bytes) 000024[1] 1781046649.409273 mapwrite 0x4cc684 thru 0x4cdad9 (0x1456 bytes) 000025[0] 1781046649.434236 read 0x32dcc2 thru 0x32de80 (0x1bf bytes) 000026[1] 1781046649.437957 mapwrite 0x0d74ff thru 0x0d7c98 (0x79a bytes) 000027[1] 1781046649.456395 read 0x09c82b thru 0x09db33 (0x1309 bytes) 000028[0] 1781046649.465702 read 0x5e42fd thru 0x5e58ba (0x15be bytes) 000030[0] 1781046649.474329 mapread 0x05fa4a thru 0x05fdd5 (0x38c bytes) 000031[0] 1781046649.486402 mapread 0x65f049 thru 0x65f7b7 (0x76f bytes) 000032[1] 1781046649.492644 trunc from 0x7b3c34 to 0x70ac6f (0xa8fc4 bytes) 000034[1] 1781046649.510902 mapread 0x6c4090 thru 0x6c519f (0x1110 bytes) 000035[1] 1781046649.550402 read 0x2d0126 thru 0x2d1820 (0x16fb bytes) 000036[0] 1781046649.550825 mapread 0x3c9bc0 thru 0x3c9e73 (0x2b4 bytes) 000037[1] 1781046649.562825 write 0x16a397 thru 0x16c2dd (0x1f47 bytes) 000038[1] 1781046649.571328 write 0x4d72dd thru 0x4d74ff (0x223 bytes) truncating to largest ever: 0x88415a 000040[0] 1781046649.572300 trunc from 0x70ac70 to 0x884159 (0x1794ea bytes) 000041[1] 1781046649.591143 read 0x2cdf62 thru 0x2ce1d2 (0x271 bytes) 000042[0] 1781046649.598266 read 0x362acc thru 0x3641d8 (0x170d bytes) 000045[0] 1781046649.627428 mapread 0x13eaf3 thru 0x13f8af (0xdbd bytes) 000046[0] 1781046649.634300 write 0x11034d thru 0x110773 (0x427 bytes) 000047[0] 1781046649.638952 write 0x3afd33 thru 0x3b0a94 (0xd62 bytes) 000049[1] 1781046649.646319 read 0x576501 thru 0x577856 (0x1356 bytes) 000050[0] 1781046649.672422 mapwrite 0x3c41c2 thru 0x3c459a (0x3d9 bytes) 000051[0] 1781046649.694562 read 0x788036 thru 0x7885e1 (0x5ac bytes) 000052[1] 1781046649.700303 mapread 0x53fd92 thru 0x540508 (0x777 bytes) 000055[1] 1781046649.708661 mapread 0x76e053 thru 0x770022 (0x1fd0 bytes) 000058[0] 1781046649.712768 trunc from 0x88415a to 0x2e2aee (0x5a166b bytes) 000061[1] 1781046649.729435 write 0x0f8d4f thru 0x0fa3ce (0x1680 bytes) 000062[0] 1781046649.738149 mapwrite 0x16638d thru 0x166d01 (0x975 bytes) 000065[1] 1781046649.761937 read 0x038a83 thru 0x038f22 (0x4a0 bytes) 000066[1] 1781046649.772355 mapwrite 0x0b24e3 thru 0x0b41e9 (0x1d07 bytes) 000067[0] 1781046649.790399 read 0x04801c thru 0x048597 (0x57c bytes) 000068[0] 1781046649.819054 read 0x08ff5b thru 0x091ca1 (0x1d47 bytes) 000069[1] 1781046649.823641 write 0x5409c6 thru 0x54109e (0x6d9 bytes) 000070[0] 1781046649.830915 trunc from 0x54109f to 0x22e303 (0x312d9b bytes) 000071[1] 1781046649.846343 mapread 0x08e5d8 thru 0x08e604 (0x2d bytes) 000072[0] 1781046649.855382 write 0x3722a8 thru 0x3730a9 (0xe02 bytes) 000074[0] 1781046649.861844 mapread 0x0d41d1 thru 0x0d49b0 (0x7e0 bytes) 000075[0] 1781046649.866622 mapwrite 0x2874cb thru 0x288ad0 (0x1606 bytes) 000077[1] 1781046649.886614 read 0x083409 thru 0x085229 (0x1e21 bytes) 000079[0] 1781046649.900277 read 0x29d7d7 thru 0x29e571 (0xd9b bytes) 000081[0] 1781046649.910009 write 0x697fd1 thru 0x698c7a (0xcaa bytes) 000087[0] 1781046649.922476 read 0x4f4813 thru 0x4f630c (0x1afa bytes) 000088[1] 1781046649.935601 mapwrite 0x4f2311 thru 0x4f2d09 (0x9f9 bytes) 000089[0] 1781046649.956320 mapread 0x38b9fc thru 0x38d4bb (0x1ac0 bytes) 000091[1] 1781046649.971795 mapread 0x22564d thru 0x226105 (0xab9 bytes) 000092[1] 1781046649.978625 read 0x4d94dc thru 0x4da36f (0xe94 bytes) 000093[1] 1781046649.984451 read 0x024217 thru 0x0259e8 (0x17d2 bytes) 000095[1] 1781046649.997140 trunc from 0x698c7b to 0x1e8d9e (0x4afedc bytes) 000097[0] 1781046650.013243 mapread 0x169089 thru 0x169f7d (0xef5 bytes) 000098[0] 1781046650.021701 mapwrite 0x1e6908 thru 0x1e6e4a (0x543 bytes) 000099[1] 1781046650.042292 mapread 0x0215b3 thru 0x023241 (0x1c8f bytes) 000102[1] 1781046650.058156 mapwrite 0x923995 thru 0x924310 (0x97c bytes) 000103[1] 1781046650.124771 read 0x011fb8 thru 0x013d4c (0x1d95 bytes) 000104[1] 1781046650.133682 read 0x8213f5 thru 0x821475 (0x81 bytes) 000105[0] 1781046650.137617 trunc from 0x924311 to 0x6ae04e (0x2762c2 bytes) 000108[1] 1781046650.162670 read 0x5be593 thru 0x5be832 (0x2a0 bytes) 000111[1] 1781046650.169276 read 0x587478 thru 0x588e19 (0x19a2 bytes) 000112[0] 1781046650.175527 mapwrite 0x112604 thru 0x1134a7 (0xea4 bytes) 000113[1] 1781046650.191563 trunc from 0x6ae04f to 0x3f7a78 (0x2b65d6 bytes) 000115[0] 1781046650.207905 mapread 0x2aae50 thru 0x2acc66 (0x1e17 bytes) 000117[0] 1781046650.217160 trunc from 0x3f7a79 to 0x21add6 (0x1dcca2 bytes) 000118[1] 1781046650.229141 mapwrite 0x1c9881 thru 0x1ca755 (0xed5 bytes) 000119[1] 1781046650.250619 trunc from 0x21add7 to 0x1e9581 (0x31855 bytes) 000122[0] 1781046650.266369 mapwrite 0x30c3a8 thru 0x30cf96 (0xbef bytes) 000123[1] 1781046650.288028 read 0x152c0c thru 0x15458f (0x1984 bytes) 000124[0] 1781046650.297581 read 0x1e32c1 thru 0x1e339c (0xdc bytes) 000126[0] 1781046650.308052 write 0x745ef5 thru 0x746cef (0xdfb bytes) 000128[0] 1781046650.316843 read 0x0ae840 thru 0x0afd27 (0x14e8 bytes) 000129[1] 1781046650.342581 mapwrite 0x0559ef thru 0x057918 (0x1f2a bytes) 000131[0] 1781046650.380034 write 0x967cf6 thru 0x96934f (0x165a bytes) 000134[0] 1781046650.392131 mapwrite 0x7e45da thru 0x7e5c96 (0x16bd bytes) 000137[0] 1781046650.422481 mapwrite 0x7b81fd thru 0x7b9d92 (0x1b96 bytes) 000138[0] 1781046650.439334 mapread 0x76cea0 thru 0x76edd7 (0x1f38 bytes) 000141[1] 1781046650.451603 read 0x8c65d4 thru 0x8c6f65 (0x992 bytes) 000143[1] 1781046650.463887 write 0x75fb8e thru 0x75ff31 (0x3a4 bytes) 000144[1] 1781046650.479601 write 0x51b9cb thru 0x51c3df (0xa15 bytes) 000145[0] 1781046650.485875 mapwrite 0x7e42a6 thru 0x7e59b5 (0x1710 bytes) 000149[1] 1781046650.519475 read 0x5b8c91 thru 0x5b988c (0xbfc bytes) 000150[0] 1781046650.549836 mapread 0x15baf8 thru 0x15d262 (0x176b bytes) 000155[0] 1781046650.566606 read 0x3f0039 thru 0x3f11f1 (0x11b9 bytes) 000157[1] 1781046650.573080 write 0x6556bd thru 0x657440 (0x1d84 bytes) 000158[0] 1781046650.577331 mapwrite 0x07958c thru 0x07a118 (0xb8d bytes) 000160[0] 1781046650.633439 mapread 0x8d8c1c thru 0x8da6c7 (0x1aac bytes) 000161[1] 1781046650.633944 mapwrite 0x02e378 thru 0x02f900 (0x1589 bytes) 000163[0] 1781046650.655941 mapread 0x1a8de6 thru 0x1aa08b (0x12a6 bytes) 000164[0] 1781046650.663612 write 0x6d7541 thru 0x6d8280 (0xd40 bytes) 000165[0] 1781046650.667692 mapread 0x78a85c thru 0x78c832 (0x1fd7 bytes) 000167[1] 1781046650.668311 mapwrite 0x3eedc1 thru 0x3f0444 (0x1684 bytes) 000171[1] 1781046650.705999 trunc from 0x969350 to 0x61b4f7 (0x34de58 bytes) 000173[0] 1781046650.719055 read 0x0fe9d1 thru 0x0ff974 (0xfa4 bytes) 000175[1] 1781046650.729169 read 0x3b716e thru 0x3b872e (0x15c1 bytes) 000184[1] 1781046650.744783 read 0x0d8de4 thru 0x0d9ecc (0x10e9 bytes) 000188[1] 1781046650.783581 read 0x217e4d thru 0x218bb3 (0xd67 bytes) 000190[1] 1781046650.790382 write 0x7bc36d thru 0x7be1be (0x1e52 bytes) truncating to largest ever: 0x8b5f14 000191[1] 1781046650.794286 trunc from 0x7be1bf to 0x8b5f13 (0xf7d55 bytes) 000194[0] 1781046650.812270 read 0x301395 thru 0x302414 (0x1080 bytes) 000197[0] 1781046650.834839 write 0x33f899 thru 0x3402c7 (0xa2f bytes) 000198[0] 1781046650.844075 mapwrite 0x3ae726 thru 0x3afa7c (0x1357 bytes) 000200[0] 1781046650.865261 write 0x536e7e thru 0x536f73 (0xf6 bytes) 000201[1] 1781046650.869992 trunc from 0x8b5f14 to 0x67c9fe (0x239515 bytes) 000202[1] 1781046650.880568 mapwrite 0x1235c3 thru 0x125578 (0x1fb6 bytes) 000204[0] 1781046650.898737 read 0x39c4eb thru 0x39ce58 (0x96e bytes) 000206[0] 1781046650.909177 mapread 0x240990 thru 0x2422ef (0x1960 bytes) 000207[0] 1781046650.931653 write 0x42a9bb thru 0x42abb8 (0x1fe bytes) 000208[0] 1781046650.940371 write 0x7d9313 thru 0x7d94f2 (0x1e0 bytes) 000209[0] 1781046650.948391 trunc from 0x7d94f3 to 0x876bdf (0x9d6ed bytes) truncating to largest ever: 0x932aba 000210[0] 1781046650.956502 trunc from 0x876be0 to 0x932ab9 (0xbbeda bytes) 000213[0] 1781046650.968964 mapread 0x1c9d45 thru 0x1ca49d (0x759 bytes) 000216[0] 1781046650.997405 mapread 0x3e48a4 thru 0x3e660b (0x1d68 bytes) 000217[0] 1781046651.000726 mapwrite 0x0cd2d9 thru 0x0cee75 (0x1b9d bytes) 000218[1] 1781046651.013733 mapread 0x6a7d50 thru 0x6a8ac3 (0xd74 bytes) 000221[1] 1781046651.029553 mapread 0x169d8b thru 0x16bce5 (0x1f5b bytes) 000224[1] 1781046651.039246 write 0x39d641 thru 0x39e662 (0x1022 bytes) 000225[0] 1781046651.043542 mapwrite 0x872286 thru 0x8739be (0x1739 bytes) 000226[0] 1781046651.064691 write 0x132218 thru 0x132fbe (0xda7 bytes) 000228[1] 1781046651.070177 mapread 0x765842 thru 0x765a0c (0x1cb bytes) 000229[0] 1781046651.080088 trunc from 0x932aba to 0x29d521 (0x695598 bytes) 000231[0] 1781046651.088057 mapread 0x0329b9 thru 0x0346c2 (0x1d0a bytes) 000232[1] 1781046651.096486 trunc from 0x29d522 to 0x67c4a0 (0x3def7f bytes) truncating to largest ever: 0x959a2f 000233[0] 1781046651.108760 trunc from 0x67c4a1 to 0x959a2e (0x2dd58e bytes) 000234[0] 1781046651.116592 trunc from 0x959a2f to 0x23ac09 (0x71ee25 bytes) 000237[1] 1781046651.124618 trunc from 0x23ac0a to 0x190740 (0xaa4c9 bytes) 000238[1] 1781046651.132634 mapwrite 0x39eaca thru 0x3a0542 (0x1a79 bytes) 000240[0] 1781046651.156485 trunc from 0x3a0543 to 0x6706bb (0x2d0179 bytes) 000241[1] 1781046651.169301 mapread 0x49d29e thru 0x49f187 (0x1eea bytes) 000243[0] 1781046651.182149 write 0x709453 thru 0x70aba5 (0x1753 bytes) 000246[1] 1781046651.190152 mapread 0x6d0f52 thru 0x6d29f8 (0x1aa7 bytes) 000248[0] 1781046651.204199 trunc from 0x70aba6 to 0x459104 (0x2b1aa1 bytes) 000249[0] 1781046651.211283 read 0x149695 thru 0x14b02b (0x1997 bytes) 000250[0] 1781046651.233326 trunc from 0x459105 to 0x7df12a (0x386026 bytes) 000251[1] 1781046651.241071 read 0x0b304d thru 0x0b4934 (0x18e8 bytes) 000252[0] 1781046651.250517 mapwrite 0x49a8c0 thru 0x49b7f6 (0xf37 bytes) 000254[1] 1781046651.292638 read 0x184477 thru 0x185121 (0xcab bytes) 000258[0] 1781046651.307618 trunc from 0x7df12b to 0x103975 (0x6db7b5 bytes) 000259[0] 1781046651.321412 write 0x1d18aa thru 0x1d2014 (0x76b bytes) 000260[1] 1781046651.323446 read 0x1401fb thru 0x141a39 (0x183f bytes) 000262[1] 1781046651.338523 read 0x197496 thru 0x198e41 (0x19ac bytes) 000263[1] 1781046651.341880 write 0x437ec5 thru 0x4398d1 (0x1a0d bytes) 000266[1] 1781046651.353816 mapread 0x1fa223 thru 0x1fbb7e (0x195c bytes) 000267[0] 1781046651.356390 write 0x8699d7 thru 0x86a5d7 (0xc01 bytes) 000270[0] 1781046651.368379 mapread 0x11bab4 thru 0x11d16b (0x16b8 bytes) 000272[1] 1781046651.372234 write 0x85df85 thru 0x85ea46 (0xac2 bytes) 000275[1] 1781046651.381385 mapwrite 0x21b6b6 thru 0x21d032 (0x197d bytes) 000276[1] 1781046651.397577 read 0x6c6fa8 thru 0x6c7c99 (0xcf2 bytes) 000283[1] 1781046651.425897 write 0x1e8fc4 thru 0x1e9e34 (0xe71 bytes) 000285[1] 1781046651.436568 trunc from 0x86a5d8 to 0x5ebb83 (0x27ea54 bytes) 000288[0] 1781046651.452540 mapwrite 0x6dea59 thru 0x6dfa9e (0x1046 bytes) truncating to largest ever: 0x9e5e99 000289[1] 1781046651.474570 trunc from 0x6dfa9f to 0x9e5e98 (0x3063fa bytes) 000290[0] 1781046651.487127 read 0x91ed3d thru 0x91f3dd (0x6a1 bytes) truncating to largest ever: 0x9ec7c8 000291[0] 1781046651.498812 trunc from 0x9e5e99 to 0x9ec7c7 (0x692f bytes) 000292[1] 1781046651.509740 read 0x179dc0 thru 0x17ae73 (0x10b4 bytes) 000296[0] 1781046651.532300 write 0x89e473 thru 0x89ff7c (0x1b0a bytes) 000297[0] 1781046651.539345 mapread 0x8a73b4 thru 0x8a88b0 (0x14fd bytes) 000298[1] 1781046651.542137 mapread 0x3a3fa7 thru 0x3a55c5 (0x161f bytes) 000299[1] 1781046651.551261 mapwrite 0x90659c thru 0x908296 (0x1cfb bytes) 000301[1] 1781046651.580723 write 0x22c389 thru 0x22d43e (0x10b6 bytes) 000302[0] 1781046651.635998 read 0x993f65 thru 0x995238 (0x12d4 bytes) 000304[1] 1781046651.646821 mapread 0x247c00 thru 0x247db9 (0x1ba bytes) 000305[0] 1781046651.647358 mapwrite 0x2670eb thru 0x267c52 (0xb68 bytes) 000307[1] 1781046651.676436 mapread 0x4400c8 thru 0x440880 (0x7b9 bytes) 000308[0] 1781046651.681642 mapwrite 0x969fcc thru 0x96b95c (0x1991 bytes) 000309[1] 1781046651.703251 mapwrite 0x376e3e thru 0x376ff5 (0x1b8 bytes) 000310[1] 1781046651.720434 mapread 0x7e6754 thru 0x7e7de4 (0x1691 bytes) 000313[0] 1781046651.731359 mapread 0x289918 thru 0x28a2cc (0x9b5 bytes) 000315[1] 1781046651.740067 trunc from 0x9ec7c8 to 0x8af2bc (0x13d50b bytes) 000316[1] 1781046651.747161 trunc from 0x8af2bd to 0x18821e (0x72709e bytes) 000320[0] 1781046651.756151 read 0x02091c thru 0x020e6e (0x553 bytes) 000321[1] 1781046651.768940 trunc from 0x18821f to 0x498b53 (0x310935 bytes) 000325[1] 1781046651.779307 trunc from 0x498b54 to 0x40b962 (0x8d1f1 bytes) 000328[0] 1781046651.783856 write 0x8f7e92 thru 0x8f9456 (0x15c5 bytes) 000329[0] 1781046651.792228 write 0x449a57 thru 0x44a35a (0x904 bytes) 000330[1] 1781046651.799447 mapread 0x309104 thru 0x309897 (0x794 bytes) 000334[1] 1781046651.817466 trunc from 0x8f9457 to 0x7d23d2 (0x127084 bytes) 000335[1] 1781046651.826837 write 0x72c309 thru 0x72db21 (0x1819 bytes) 000336[0] 1781046651.834320 mapread 0x30a12a thru 0x30c04f (0x1f26 bytes) 000338[1] 1781046651.854507 read 0x068968 thru 0x069718 (0xdb1 bytes) 000340[0] 1781046651.894502 trunc from 0x7d23d3 to 0x73bc3c (0x96796 bytes) 000341[0] 1781046651.919308 read 0x258ec1 thru 0x259ea2 (0xfe2 bytes) 000342[1] 1781046651.948592 mapread 0x392b97 thru 0x3934bd (0x927 bytes) 000345[0] 1781046651.964277 mapwrite 0x404f29 thru 0x405d89 (0xe61 bytes) 000346[1] 1781046651.975739 read 0x188d57 thru 0x189ed4 (0x117e bytes) 000349[1] 1781046651.988590 mapwrite 0x88573b thru 0x8871ec (0x1ab2 bytes) 000352[0] 1781046652.058330 read 0x3977a4 thru 0x398d2b (0x1588 bytes) 000356[1] 1781046652.079247 mapread 0x7f2f94 thru 0x7f4f85 (0x1ff2 bytes) 000357[1] 1781046652.083195 read 0x6c848d thru 0x6c895f (0x4d3 bytes) 000359[1] 1781046652.093066 trunc from 0x8871ed to 0x98d539 (0x10634d bytes) 000360[0] 1781046652.102640 mapread 0x30b799 thru 0x30c9c7 (0x122f bytes) 000361[0] 1781046652.113307 mapread 0x5e8019 thru 0x5e8cbd (0xca5 bytes) 000362[1] 1781046652.119538 write 0x21c0e0 thru 0x21cbe1 (0xb02 bytes) 000364[0] 1781046652.123937 read 0x18dada thru 0x18ed76 (0x129d bytes) 000365[1] 1781046652.128552 read 0x116eea thru 0x1186a9 (0x17c0 bytes) 000366[0] 1781046652.159711 mapwrite 0x9e2894 thru 0x9e4410 (0x1b7d bytes) 000367[1] 1781046652.240597 mapwrite 0x7edc82 thru 0x7ef202 (0x1581 bytes) 000368[1] 1781046652.271891 read 0x39640f thru 0x397de6 (0x19d8 bytes) 000369[0] 1781046652.280796 mapread 0x2dda4d thru 0x2ddeba (0x46e bytes) 000370[0] 1781046652.287950 read 0x6ea702 thru 0x6ebf90 (0x188f bytes) 000371[0] 1781046652.292842 trunc from 0x9e4411 to 0x7f5f18 (0x1ee4f8 bytes) 000372[1] 1781046652.310321 trunc from 0x7f5f19 to 0x94ab5b (0x154c43 bytes) 000373[1] 1781046652.327852 mapwrite 0x0915ce thru 0x092389 (0xdbc bytes) 000375[1] 1781046652.414170 write 0x0d189c thru 0x0d3318 (0x1a7d bytes) 000377[1] 1781046652.421768 mapread 0x022de9 thru 0x023663 (0x87b bytes) 000378[0] 1781046652.426862 write 0x8d2903 thru 0x8d44c1 (0x1bbf bytes) 000381[1] 1781046652.463388 write 0x58f497 thru 0x591034 (0x1b9e bytes) 000382[0] 1781046652.471429 trunc from 0x94ab5c to 0x1eb161 (0x75f9fa bytes) 000384[0] 1781046652.500164 mapwrite 0x7f37d8 thru 0x7f4fba (0x17e3 bytes) 000387[0] 1781046652.541815 mapread 0x5891a6 thru 0x58af7a (0x1dd5 bytes) 000388[0] 1781046652.545926 write 0x2702e5 thru 0x2714a1 (0x11bd bytes) 000389[1] 1781046652.555731 write 0x240b8b thru 0x241c70 (0x10e6 bytes) 000390[1] 1781046652.569572 mapread 0x4546cc thru 0x45506e (0x9a3 bytes) 000395[1] 1781046652.607941 write 0x655102 thru 0x6570d5 (0x1fd4 bytes) 000398[0] 1781046652.631139 mapread 0x05d183 thru 0x05d7b1 (0x62f bytes) 000399[1] 1781046652.651200 mapread 0x20408d thru 0x20440c (0x380 bytes) 000400[0] 1781046652.664372 read 0x2f9ba7 thru 0x2fb7f9 (0x1c53 bytes) 000401[1] 1781046652.711417 trunc from 0x7f4fbb to 0x8393a1 (0x443e7 bytes) 000403[1] 1781046652.752710 mapread 0x303cdf thru 0x304e3c (0x115e bytes) 000404[1] 1781046652.760773 write 0x2876b5 thru 0x288783 (0x10cf bytes) 000405[0] 1781046652.774798 trunc from 0x8393a2 to 0x72d5f3 (0x10bdae bytes) 000411[1] 1781046652.831765 mapread 0x4bee15 thru 0x4c069d (0x1889 bytes) 000414[1] 1781046652.851465 write 0x397979 thru 0x397b49 (0x1d1 bytes) 000418[1] 1781046652.867784 read 0x640a48 thru 0x64133f (0x8f8 bytes) 000421[0] 1781046652.898235 trunc from 0x72d5f4 to 0x0ad763 (0x67fe90 bytes) 000422[0] 1781046652.919891 write 0x917d55 thru 0x919236 (0x14e2 bytes) 000423[1] 1781046652.933670 trunc from 0x919237 to 0x407452 (0x511de4 bytes) 000424[0] 1781046652.952729 mapread 0x0cd9a0 thru 0x0cf097 (0x16f8 bytes) 000426[1] 1781046652.970629 write 0x676ae8 thru 0x678958 (0x1e71 bytes) 000428[1] 1781046652.979488 write 0x51e8fa thru 0x51f3d7 (0xade bytes) truncating to largest ever: 0x9f4917 000429[1] 1781046652.982938 trunc from 0x678959 to 0x9f4916 (0x37bfbe bytes) 000434[1] 1781046653.008510 mapwrite 0x509e76 thru 0x50a73a (0x8c5 bytes) 000436[0] 1781046653.047497 trunc from 0x9f4917 to 0x3a3079 (0x65189d bytes) 000437[0] 1781046653.060618 trunc from 0x3a307a to 0x392120 (0x10f59 bytes) 000439[0] 1781046653.069440 write 0x360054 thru 0x361039 (0xfe6 bytes) 000440[0] 1781046653.071844 mapwrite 0x823816 thru 0x82419f (0x98a bytes) 000442[1] 1781046653.096630 write 0x02e244 thru 0x02fd8d (0x1b4a bytes) 000444[0] 1781046653.107494 trunc from 0x8241a0 to 0x43da58 (0x3e6747 bytes) 000445[0] 1781046653.119593 mapwrite 0x444793 thru 0x445d98 (0x1606 bytes) 000446[1] 1781046653.137423 mapwrite 0x1e75e0 thru 0x1e84e2 (0xf03 bytes) 000448[1] 1781046653.162927 read 0x33d3c7 thru 0x33e6f4 (0x132e bytes) 000449[0] 1781046653.173461 trunc from 0x445d99 to 0x4e14a4 (0x9b70c bytes) 000452[0] 1781046653.186638 read 0x1cef20 thru 0x1cf7b6 (0x897 bytes) 000453[1] 1781046653.209486 mapread 0x1a6dea thru 0x1a7891 (0xaa8 bytes) 000455[0] 1781046653.223219 read 0x291d9f thru 0x29370c (0x196e bytes) 000458[0] 1781046653.228035 mapread 0x3c8c78 thru 0x3ca306 (0x168f bytes) 000459[1] 1781046653.233256 trunc from 0x4e14a5 to 0x691dbf (0x1b091b bytes) 000464[1] 1781046653.260144 mapwrite 0x46d835 thru 0x46eb48 (0x1314 bytes) 000466[1] 1781046653.300449 mapwrite 0x469171 thru 0x469d53 (0xbe3 bytes) 000468[0] 1781046653.315002 write 0x5047a7 thru 0x504b11 (0x36b bytes) 000469[0] 1781046653.323011 read 0x580bff thru 0x5815d3 (0x9d5 bytes) 000470[1] 1781046653.330404 read 0x5da129 thru 0x5db42f (0x1307 bytes) 000471[1] 1781046653.340408 mapwrite 0x349010 thru 0x34a705 (0x16f6 bytes) 000474[0] 1781046653.362422 write 0x93258c thru 0x932ecb (0x940 bytes) 000475[1] 1781046653.369402 write 0x7947a0 thru 0x7956e9 (0xf4a bytes) 000477[0] 1781046653.389842 mapread 0x7132d0 thru 0x713e6d (0xb9e bytes) 000479[1] 1781046653.402420 write 0x0ce8da thru 0x0ceda6 (0x4cd bytes) 000480[0] 1781046653.406866 mapread 0x916360 thru 0x91800c (0x1cad bytes) 000482[0] 1781046653.418115 read 0x34b0c8 thru 0x34b999 (0x8d2 bytes) 000483[1] 1781046653.424328 read 0x30fc2a thru 0x3104b5 (0x88c bytes) 000484[0] 1781046653.443909 trunc from 0x932ecc to 0x628ef9 (0x309fd2 bytes) 000485[0] 1781046653.466655 trunc from 0x628efa to 0x8267c1 (0x1fd8c8 bytes) 000487[0] 1781046653.475624 mapread 0x25655b thru 0x2566fb (0x1a1 bytes) 000488[0] 1781046653.507386 mapread 0x038c8f thru 0x039c4a (0xfbc bytes) 000489[0] 1781046653.513420 mapread 0x773b52 thru 0x77453a (0x9e9 bytes) 000490[0] 1781046653.513945 trunc from 0x8267c2 to 0x25751e (0x5cf2a3 bytes) 000496[1] 1781046653.533483 trunc from 0x25751f to 0x72f754 (0x4d8236 bytes) 000497[0] 1781046653.546332 write 0x393a14 thru 0x3947bf (0xdac bytes) 000498[1] 1781046653.553712 mapread 0x21ad28 thru 0x21bb6b (0xe44 bytes) 000499[1] 1781046653.562262 read 0x03e76a thru 0x03f606 (0xe9d bytes) All operations completed A-OK! PASS 16b (6s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 19:10:55 (1781046655) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (2s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 19:10:56 (1781046656) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0808041 s, 26.0 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0565399 s, 37.1 MB/s ldlm.namespaces.MGC192.168.203.105@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff975f46293000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff975f48093800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff975f48093800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff975f46293000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff975f48093800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.28593 s, 45.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.586851 s, 179 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.678852 s, 154 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.31585 s, 79.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.44883 s, 72.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.922444 s, 114 MB/s PASS 16d (10s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 19:11:06 (1781046666) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.0874811 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288731 s, 36.3 MB/s PASS 16e (2s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 19:11:09 (1781046669) TIMEOUT PASS 16f (22s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 19:11:31 (1781046691) TIMEOUT PASS 16g (22s) == sanityn test 16i: read after truncate file ============ 19:11:53 (1781046713) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (2s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 19:11:55 (1781046715) fail_loc=0x8000030a PASS 17 (4s) == sanityn test 18: mmap sanity check =========================================================================================== 19:12:00 (1781046720) mmap test1: basic mmap operation (PASS, 0.016384s) mmap test2: MAP_PRIVATE not write back (PASS, 0.015693s) mmap test3: concurrent mmap ops on two nodes (PASS, 2.7711s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.1159s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.027136s) mmap test6: check mmap write/read content on two nodes (PASS, 0.029221s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0135s) PASS 18 (8s) == sanityn test 19: test concurrent uncached read races ========================================================================= 19:12:09 (1781046729) oleg305-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg305-client: oleg305-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 19:12:10 (1781046730) PASS 20 (2s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 19:12:12 (1781046732) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) == sanityn test 23: others should see updated atime while another read============================================================== 19:12:14 (1781046734) now is 1781046735 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 new atime is 1781046796 PASS 23 (63s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 19:13:17 (1781046797) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.3K 376 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.2K 369 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 376 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 369 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 5.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 11.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118072 376 117696 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235729 369 235360 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 376 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 369 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118072 376 117696 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre[OST:1] filesystem_summary: 235729 369 235360 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118072 376 117696 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235729 369 235360 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 PASS 24a (2s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 19:13:19 (1781046799) PASS 24b (2s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 19:13:21 (1781046801) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (3s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 19:13:24 (1781046804) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 19:13:25 (1781046805) utime: good mknod times 1781046804 <= 1781046805 <= 1781046805 for /mnt/lustre/f26a utime: good mknod times 1781046804 <= 1781046805 <= 1781046805 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (3s) == sanityn test 26b: sync mtime between ost and mds ====== 19:13:28 (1781046808) PASS 26b (3s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 19:13:32 (1781046812) 0 PASS 30 (4s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 19:13:36 (1781046816) fail_loc=0x314 PASS 31a (3s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 19:13:39 (1781046819) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg305-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.203.105@tcp:/lustre 7542784 12288 7524352 1% /mnt/lustre2 PASS 31b (13s) == sanityn test 31r: open-rename(replace) race =========== 19:13:52 (1781046832) -rw-r--r-- 1 root root 0 Jun 9 19:13 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 mv: failed to access '/mnt/lustre2/f31r.sanityn.target': Input/output error PASS 31r (5s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 19:13:57 (1781046837) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 19:13:58 (1781046838) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 19:13:59 (1781046839) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 19:14:00 (1781046840) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 19:14:01 (1781046841) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg305-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff975f46293000.ost_server_uuid,osc.lustre-OST0000-osc-ffff975f48093800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff975f46293000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff975f48093800.ost_server_uuid in IDLE state after 0 sec oleg305-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff975f46293000.ost_server_uuid,osc.lustre-OST0001-osc-ffff975f48093800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff975f46293000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff975f48093800.ost_server_uuid in FULL state after 0 sec oleg305-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff975f46293000.ost_server_uuid,osc.lustre-OST0000-osc-ffff975f48093800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff975f46293000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff975f48093800.ost_server_uuid in IDLE state after 0 sec oleg305-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff975f46293000.ost_server_uuid,osc.lustre-OST0001-osc-ffff975f48093800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff975f46293000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff975f48093800.ost_server_uuid in FULL state after 0 sec oleg305-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff975f46293000.ost_server_uuid,osc.lustre-OST0000-osc-ffff975f48093800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff975f46293000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff975f48093800.ost_server_uuid in IDLE state after 0 sec oleg305-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff975f46293000.ost_server_uuid,osc.lustre-OST0001-osc-ffff975f48093800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff975f46293000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff975f48093800.ost_server_uuid in FULL state after 0 sec PASS 34 (78s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 19:15:19 (1781046919) fail_loc=0x80000317 Race attempt 0 Wait for 44604 44734 for 60 sec... total: 4000 open/close in 8.19 seconds: 488.15 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.203.105@tcp:/lustre 7.2G 12M 7.2G 1% /mnt/lustre 192.168.203.105@tcp:/lustre 7.2G 12M 7.2G 1% /mnt/lustre2 PASS 35 (65s) == sanityn test 36: handle ESTALE/open-unlink correctly == 19:16:24 (1781046984) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg305-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.313951 s, 167 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.378657 s, 138 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.369171 s, 142 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.358914 s, 146 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.327165 s, 160 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.371913 s, 141 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.364967 s, 144 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.34796 s, 151 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.35413 s, 148 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.342692 s, 153 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.367557 s, 143 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7526400) after_dd(7475200) after(7526400) PASS 36 (241s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 19:20:25 (1781047225) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 - create 9836 (time 1781047236.37 total 10.00 last 983.59) total: 10000 create in 10.16 seconds: 984.18 ops/second PASS 37 (14s) == sanityn test 39a: test from 11063 ============================================================================================ 19:20:39 (1781047239) repeat after cancel_lru_locks PASS 39a (3s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 19:20:42 (1781047242) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 19:20:45 (1781047245) repeat after cancel_lru_locks PASS 39c (4s) == sanityn test 39d: sync write should update mtime ====== 19:20:49 (1781047249) fail_loc=0x411 fail_loc=0 PASS 39d (2s) == sanityn test 40a: pdirops: create vs others ======================================================================== 19:20:51 (1781047251) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (9s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 19:21:00 (1781047260) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (8s) == sanityn test 40c: pdirops: link and others ======================================================================== 19:21:08 (1781047268) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (9s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 19:21:17 (1781047277) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (8s) == sanityn test 40e: pdirops: rename and others ======================================================================== 19:21:25 (1781047285) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (8s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 19:21:33 (1781047293) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (5s) == sanityn test 41b: pdirops: create vs create ======================================================================== 19:21:38 (1781047298) open: File exists Conflict PASS 41b (6s) == sanityn test 41c: pdirops: create vs link ======================================================================== 19:21:44 (1781047304) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (6s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 19:21:50 (1781047310) Conflict PASS 41d (5s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 19:21:56 (1781047316) Conflict PASS 41e (5s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 19:22:00 (1781047320) Conflict PASS 41f (6s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 19:22:06 (1781047326) Conflict PASS 41g (6s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 19:22:12 (1781047332) Conflict PASS 41h (6s) == sanityn test 41i: reint_open: create vs create ======== 19:22:18 (1781047338) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (741s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 19:34:39 (1781048079) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (6s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 19:34:45 (1781048085) open: File exists Conflict PASS 42b (5s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 19:34:50 (1781048090) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (5s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 19:34:55 (1781048095) Conflict PASS 42d (5s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 19:35:00 (1781048100) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (5s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 19:35:05 (1781048105) Conflict PASS 42f (4s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 19:35:09 (1781048109) Conflict PASS 42g (5s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 19:35:14 (1781048114) Conflict PASS 42h (5s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 19:35:19 (1781048119) PASS 43a (11s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 19:35:30 (1781048130) Conflict PASS 43b (5s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 19:35:35 (1781048135) Conflict PASS 43c (4s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 19:35:39 (1781048139) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (5s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 19:35:44 (1781048144) Conflict PASS 43e (6s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 19:35:50 (1781048150) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (5s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 19:35:55 (1781048155) stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (5s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 19:36:00 (1781048160) Conflict PASS 43h (5s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 19:36:05 (1781048165) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 19:36:06 (1781048166) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (52s) == sanityn test 43k: unlink vs create ==================== 19:36:58 (1781048218) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (447s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 19:44:25 (1781048665) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Tue Jun 9 19:44:28 EDT 2026 PASS 44a (5s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 19:44:30 (1781048670) open: File exists Conflict PASS 44b (4s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 19:44:35 (1781048675) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (4s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 19:44:39 (1781048679) Conflict PASS 44d (5s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 19:44:44 (1781048684) Conflict PASS 44e (5s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 19:44:49 (1781048689) Conflict PASS 44f (4s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 19:44:53 (1781048693) Conflict PASS 44g (5s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 19:44:58 (1781048698) Conflict PASS 44h (6s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 19:45:04 (1781048704) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 19:45:05 (1781048705) PASS 45a (21s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 19:45:26 (1781048726) Conflict PASS 45b (5s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 19:45:31 (1781048731) Conflict PASS 45c (5s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 19:45:36 (1781048736) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (5s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 19:45:41 (1781048741) Conflict PASS 45e (7s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 19:45:48 (1781048748) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (7s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 19:45:55 (1781048755) stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (6s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 19:46:01 (1781048761) Conflict PASS 45h (5s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 19:46:06 (1781048766) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 19:46:07 (1781048767) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (532s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 19:54:59 (1781049299) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (5s) == sanityn test 46b: pdirops: link vs create ======================================================================== 19:55:04 (1781049304) open: File exists Conflict PASS 46b (5s) == sanityn test 46c: pdirops: link vs link ======================================================================== 19:55:09 (1781049309) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (6s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 19:55:15 (1781049315) Conflict PASS 46d (5s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 19:55:20 (1781049320) Conflict PASS 46e (5s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 19:55:25 (1781049325) Conflict PASS 46f (5s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 19:55:30 (1781049330) Conflict PASS 46g (5s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 19:55:35 (1781049335) Conflict PASS 46h (5s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 19:55:40 (1781049340) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 19:55:41 (1781049341) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 19:55:42 (1781049342) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 19:55:43 (1781049343) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 19:55:44 (1781049344) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 19:55:45 (1781049345) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 19:55:46 (1781049346) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 19:55:47 (1781049347) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 19:55:48 (1781049348) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.00498678 s, 2.1 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 19:55:55 (1781049355) 0+1 records in 0+1 records out 158 bytes copied, 0.00351089 s, 45.0 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (242692) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 19:55:59 (1781049359) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00258496 s, 396 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150445 s, 69.7 MB/s 3145728 PASS 51b (15s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 19:56:14 (1781049374) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00358039 s, 286 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 19:56:20 (1781049380) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122719 s, 85.4 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (3s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 19:56:23 (1781049383) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2437 0x985 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2437 0x985 0 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 19:56:27 (1781049387) llite.lustre-ffff975f46293000.xattr_cache=1 llite.lustre-ffff975f48093800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504962 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-09 19:56:28.000000000 -0400 Modify: 2026-06-09 19:56:28.000000000 -0400 Change: 2026-06-09 19:56:28.000000000 -0400 Birth: 2026-06-09 19:56:28.000000000 -0400 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504965 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-09 19:56:33.000000000 -0400 Modify: 2026-06-09 19:56:34.000000000 -0400 Change: 2026-06-09 19:56:34.000000000 -0400 Birth: 2026-06-09 19:56:33.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-09 19:56:39.000000000 -0400 Modify: 2026-06-09 19:56:39.000000000 -0400 Change: 2026-06-09 19:56:39.000000000 -0400 Birth: 2026-06-09 19:56:39.000000000 -0400 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504967 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-09 19:56:39.000000000 -0400 Modify: 2026-06-09 19:56:39.000000000 -0400 Change: 2026-06-09 19:56:39.000000000 -0400 Birth: 2026-06-09 19:56:39.000000000 -0400 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504970 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-09 19:56:44.000000000 -0400 Modify: 2026-06-09 19:56:45.000000000 -0400 Change: 2026-06-09 19:56:45.000000000 -0400 Birth: 2026-06-09 19:56:44.000000000 -0400 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-09 19:56:44.000000000 -0400 Modify: 2026-06-09 19:56:45.000000000 -0400 Change: 2026-06-09 19:56:45.000000000 -0400 Birth: 2026-06-09 19:56:44.000000000 -0400 PASS 54 (24s) == sanityn test 55a: rename vs unlink target dir ========= 19:56:51 (1781049411) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (8s) == sanityn test 55b: rename vs unlink source dir ========= 19:56:59 (1781049419) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (8s) == sanityn test 55c: rename vs unlink orphan target dir == 19:57:07 (1781049427) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 19:57:20 (1781049440) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (10s) == sanityn test 60: Verify data_version behaviour ======== 19:57:29 (1781049449) PASS 60 (2s) == sanityn test 70a: cd directory && rm directory ======== 19:57:31 (1781049451) PASS 70a (2s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 19:57:33 (1781049453) PASS 70b (3s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 19:57:36 (1781049456) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0033228 s, 19.7 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 19:57:37 (1781049457) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00434747 s, 15.1 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 19:57:38 (1781049458) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 19:57:39 (1781049459) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00650988 s, 10.1 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (2s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 19:57:41 (1781049461) llite.lustre-ffff975f46293000.xattr_cache=1 llite.lustre-ffff975f48093800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 19:57:43 (1781049463) llite.lustre-ffff975f46293000.xattr_cache=1 llite.lustre-ffff975f48093800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (2s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 19:57:45 (1781049465) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00278829 s, 367 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0049682 s, 206 kB/s 252724: get lock2 252725: get lock1 252724: done 252724: get lock1 252725: done 252725: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 252725: cannot get lock2: Resource deadlock avoided 252725: exit rc=0 fcntl cmd 7 failed: Resource deadlock avoided 252724: cannot get lock1: Resource deadlock avoided 252724: done 252724: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 19:57:51 (1781049471) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0192666 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112125 s, 93.5 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 253299 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 19:57:58 (1781049478) opening files: ........ closing files: ........ PASS 76 (18s) == sanityn test 77a: check FIFO NRS policy =============== 19:58:16 (1781049496) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.167195 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274192 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321872 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295134 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0739838 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0537312 s, 19.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0774547 s, 13.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0944845 s, 11.1 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lock_unused_count=2 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0672516 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0975272 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0918019 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102852 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100486 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108183 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0949545 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104314 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106377 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019479 s, 53.8 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0151135 s, 69.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171832 s, 61.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026941 s, 38.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322634 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206479 s, 50.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265243 s, 39.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284744 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281776 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025447 s, 41.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283641 s, 37.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267638 s, 39.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0333006 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298441 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0365751 s, 28.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0362821 s, 28.9 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg305-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg305-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (3s) == sanityn test 77b: check CRR-N NRS policy ============== 19:58:19 (1781049499) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.129536 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213742 s, 49.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0477708 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435128 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0343373 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0623485 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0756583 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102437 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0892382 s, 11.8 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff975f46293000.lock_unused_count=3 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105825 s, 9.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111037 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0971048 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103982 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110334 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118194 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119088 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.1141 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792282 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152531 s, 68.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145252 s, 72.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166028 s, 63.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00378614 s, 277 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00574066 s, 183 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210615 s, 49.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00997452 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00383594 s, 273 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208647 s, 50.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635731 s, 165 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024206 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00401228 s, 261 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675603 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00393563 s, 266 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00477038 s, 220 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg305-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg305-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.115934 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025821 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306506 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0773408 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0771257 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0680971 s, 15.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0742383 s, 14.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff975f46293000.lock_unused_count=1 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0956396 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0962259 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0877056 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.098286 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0862071 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114564 s, 9.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112947 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106482 s, 9.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105514 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0977332 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158835 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196411 s, 53.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850932 s, 123 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108984 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01868 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175137 s, 59.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245171 s, 42.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231183 s, 45.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197049 s, 53.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199861 s, 52.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00392391 s, 267 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310632 s, 33.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166737 s, 62.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747572 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560914 s, 187 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128537 s, 81.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lock_unused_count=2 removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg305-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg305-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (5s) == sanityn test 77c: check ORR NRS policy ================ 19:58:24 (1781049504) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.152385 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257825 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0741267 s, 14.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.053952 s, 19.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0840596 s, 12.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0709984 s, 14.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0832401 s, 12.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff975f46293000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0781924 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0957754 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0838223 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0868954 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109187 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0804531 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0952376 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0954881 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102736 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10594 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152198 s, 68.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140191 s, 74.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147005 s, 71.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195281 s, 53.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153467 s, 68.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229949 s, 45.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220652 s, 47.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234854 s, 44.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00421281 s, 249 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168741 s, 62.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660994 s, 159 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134035 s, 78.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154282 s, 68.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201816 s, 52.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009483 s, 111 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160665 s, 65.3 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg305-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg305-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.128279 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188757 s, 55.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0486504 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0720297 s, 14.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0680573 s, 15.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0768586 s, 13.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0804087 s, 13.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0927359 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0796873 s, 13.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0740798 s, 14.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.077625 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10403 s, 10.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112259 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102405 s, 10.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lock_unused_count=7 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100691 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0879633 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.09961 s, 10.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019481 s, 53.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122111 s, 85.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189588 s, 55.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185488 s, 56.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202472 s, 51.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216639 s, 48.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130201 s, 80.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229056 s, 45.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250413 s, 41.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232571 s, 45.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00943986 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113887 s, 92.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233274 s, 45.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0063659 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110926 s, 94.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183601 s, 57.1 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg305-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg305-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.109041 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336654 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0550324 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.074317 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0851377 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0625537 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0885908 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0806678 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.097709 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0835846 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0932706 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0906799 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0718682 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102084 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.082776 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0949819 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112805 s, 9.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff975f46293000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00375003 s, 280 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108693 s, 96.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00402235 s, 261 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145183 s, 72.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123589 s, 84.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190542 s, 55.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313064 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011281 s, 93.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210183 s, 49.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103783 s, 101 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198142 s, 52.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101348 s, 103 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175782 s, 59.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146948 s, 71.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244619 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572909 s, 183 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg305-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg305-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (6s) == sanityn test 77d: check TRR nrs policy ================ 19:58:30 (1781049510) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.17778 s, 94.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0460682 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0526955 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.056754 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0382584 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0748502 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0890644 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0768169 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0737214 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0911821 s, 11.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0897339 s, 11.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff975f46293000.lock_unused_count=3 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108085 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103694 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0879753 s, 11.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120303 s, 8.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0964506 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0692097 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176535 s, 59.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177888 s, 58.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136217 s, 77.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141307 s, 74.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263267 s, 39.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154033 s, 68.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256408 s, 40.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256646 s, 40.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206599 s, 50.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110579 s, 94.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00456466 s, 230 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848614 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180567 s, 58.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165709 s, 63.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011714 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016816 s, 62.4 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg305-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg305-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.119808 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.034515 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0400923 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0355229 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0638143 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0559096 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0549852 s, 19.1 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0001-osc-ffff975f46293000.lock_unused_count=2 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0795764 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0725251 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0751731 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0692073 s, 15.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0001-osc-ffff975f46293000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0908932 s, 11.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119309 s, 8.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10556 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112343 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0963205 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115775 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207648 s, 50.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222601 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187791 s, 55.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115314 s, 90.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660598 s, 159 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146526 s, 71.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170601 s, 61.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00421762 s, 249 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00367019 s, 286 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222095 s, 47.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125465 s, 83.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024496 s, 42.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125668 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177271 s, 59.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020804 s, 50.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00445371 s, 235 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg305-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg305-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.126792 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0398146 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0595483 s, 17.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0676568 s, 15.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0853374 s, 12.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0802832 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0837217 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0684822 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0673496 s, 15.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0996031 s, 10.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106784 s, 9.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115304 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0994514 s, 10.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102591 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107766 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0967725 s, 10.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11599 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157312 s, 66.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112635 s, 93.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163714 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172334 s, 60.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185507 s, 56.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234905 s, 44.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00444623 s, 236 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218812 s, 47.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00404178 s, 259 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854212 s, 123 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246447 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224272 s, 46.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026572 s, 39.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202915 s, 51.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00510203 s, 206 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139444 s, 75.2 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg305-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg305-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (5s) == sanityn test 77e: check TBF NID nrs policy ============ 19:58:35 (1781049515) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.203.5@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.22445 s, 74.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0353473 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0605643 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0704145 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0618443 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0634268 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0773362 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0657121 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0559373 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0783499 s, 13.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0570852 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0735762 s, 14.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0799232 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0924836 s, 11.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0726411 s, 14.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0802416 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0803176 s, 13.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.013439 s, 78.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240987 s, 43.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232145 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214001 s, 49.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189541 s, 55.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169487 s, 61.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313254 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00476152 s, 220 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126937 s, 82.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00431304 s, 243 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771359 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00382682 s, 274 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00397365 s, 264 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00345506 s, 303 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00285717 s, 367 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00188807 s, 555 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg305-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg305-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.130299 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0401465 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0416944 s, 25.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0695837 s, 15.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0612397 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0819622 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0719084 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0666971 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0749221 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0829622 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0728061 s, 14.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0732278 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0804153 s, 13.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0802442 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0634194 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0797028 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0960648 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188647 s, 55.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250301 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227315 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185267 s, 56.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216634 s, 48.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192824 s, 54.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199383 s, 52.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911448 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182189 s, 57.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00898739 s, 117 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00393159 s, 267 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00502676 s, 209 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00983532 s, 107 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314372 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844478 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00392043 s, 267 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg305-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg305-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.164836 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338962 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0657716 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.06725 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0775533 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0776234 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0650483 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0695579 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0694834 s, 15.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0751756 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0811429 s, 12.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0908844 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0888586 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101458 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0697145 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0936674 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107402 s, 9.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff975f46293000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208024 s, 50.4 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170796 s, 61.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226615 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198202 s, 52.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228587 s, 45.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217626 s, 48.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173042 s, 60.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239457 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205494 s, 51.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065891 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174348 s, 60.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102581 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00391941 s, 268 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00426753 s, 246 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00394324 s, 266 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00381839 s, 275 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg305-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg305-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.121109 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204876 s, 51.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.048024 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0576014 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238507 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0464844 s, 22.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff975f46293000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0689537 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0520569 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0384544 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0670639 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0551969 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101833 s, 10.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0883448 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.066882 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12128 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10178 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111144 s, 9.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219171 s, 47.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215747 s, 48.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026403 s, 39.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212425 s, 49.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017788 s, 58.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168739 s, 62.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152189 s, 68.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127155 s, 82.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213581 s, 49.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153755 s, 68.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215846 s, 48.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0131991 s, 79.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283512 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860369 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912234 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896971 s, 117 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg305-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg305-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (9s) == sanityn test 77f: check TBF JobID nrs policy ========== 19:58:44 (1781049524) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg305-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.150537 s, 111 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316103 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0364821 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0709544 s, 14.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0759271 s, 13.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0711118 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0917809 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0650771 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0743706 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0743257 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0866832 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0917875 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.093791 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0745539 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0883253 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.086904 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0849522 s, 12.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174326 s, 60.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116886 s, 89.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181037 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158661 s, 66.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206066 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161607 s, 64.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202156 s, 51.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168165 s, 62.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.014035 s, 74.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00366131 s, 286 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247018 s, 42.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283222 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732212 s, 143 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00977337 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00418791 s, 250 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00432537 s, 242 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg305-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg305-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg305-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.152789 s, 110 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0392348 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0388127 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.051279 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276544 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0606645 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0566202 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0773755 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0789418 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0650335 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0767502 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0751986 s, 13.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10961 s, 9.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.090354 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0963706 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0799259 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.091433 s, 11.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142275 s, 73.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126342 s, 83.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0041459 s, 253 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123529 s, 84.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163928 s, 64.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127931 s, 82.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176232 s, 59.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197143 s, 53.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164727 s, 63.7 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00429763 s, 244 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825872 s, 127 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621575 s, 169 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121699 s, 86.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676473 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0018262 s, 574 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00172071 s, 609 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg305-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg305-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg305-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.16556 s, 101 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0436141 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0464994 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0537646 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0637184 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0797985 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0720054 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0798012 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0833493 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0705607 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0972694 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0946471 s, 11.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0979666 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0683466 s, 15.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0912246 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102387 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120796 s, 8.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff975f46293000.lock_unused_count=6 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135488 s, 77.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113929 s, 92.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111794 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139731 s, 75.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137061 s, 76.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173953 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161791 s, 64.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635448 s, 165 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188325 s, 55.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199815 s, 52.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00440535 s, 238 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00437952 s, 239 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205617 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715704 s, 147 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173714 s, 60.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143641 s, 73.0 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg305-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg305-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg305-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.188925 s, 88.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297929 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304223 s, 34.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021873 s, 47.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0482814 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0694543 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.062587 s, 16.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0512388 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0898304 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0907849 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0873148 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0970896 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0907922 s, 11.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103333 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115642 s, 9.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lock_unused_count=3 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.124548 s, 8.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117536 s, 8.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107452 s, 97.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119732 s, 87.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119744 s, 87.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126853 s, 82.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206402 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190348 s, 55.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167928 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158763 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176812 s, 59.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00389714 s, 269 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189799 s, 55.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196782 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00411578 s, 255 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00373903 s, 280 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00336192 s, 312 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00375686 s, 279 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg305-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg305-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (10s) == sanityn test 77g: Change TBF type directly ============ 19:58:54 (1781049534) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (4s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 19:58:58 (1781049538) oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg305-client: oleg305-server: ssh exited with exit code 2 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg305-client: oleg305-server: ssh exited with exit code 12 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg305-client: oleg305-server: ssh exited with exit code 12 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg305-client: oleg305-server: ssh exited with exit code 12 PASS 77h (4s) == sanityn test 77i: Change rank of TBF rule ============= 19:59:02 (1781049542) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (9s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 19:59:11 (1781049551) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.153003 s, 110 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259083 s, 40.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324911 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372439 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0712715 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.052534 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0652883 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0687332 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0635935 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0801596 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.081045 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0954511 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0762233 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0825446 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0855098 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0803929 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0768629 s, 13.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246551 s, 42.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208596 s, 50.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0323387 s, 32.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0127113 s, 82.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239785 s, 43.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02059 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296505 s, 35.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0423697 s, 24.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132079 s, 79.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116624 s, 89.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650548 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0153744 s, 68.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295271 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0340306 s, 30.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116804 s, 89.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105123 s, 99.7 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg305-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg305-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.95704 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4417 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (52s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 20:00:03 (1781049603) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg305-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.645313 s, 26.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155331 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143286 s, 7.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158666 s, 6.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157197 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159912 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144902 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.174876 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.161464 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.354361 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.355744 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.371582 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.364448 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.405366 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.407434 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.4004 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.416073 s, 2.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151285 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.151634 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150328 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150131 s, 7.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.155288 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153136 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.159182 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158087 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.14929 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154533 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152859 s, 6.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154259 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158105 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157955 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156329 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158332 s, 6.6 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg305-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg305-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8314 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9734 s, 5.2 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg305-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.687436 s, 24.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338887 s, 30.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0630675 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0576868 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0389031 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.051345 s, 20.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0457243 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0459444 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0516264 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.315844 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.335324 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.361309 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.380307 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.391931 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.381051 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.372265 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.386936 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0816968 s, 12.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0766266 s, 13.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0891569 s, 11.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.084035 s, 12.5 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0804835 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0898135 s, 11.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0913918 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0822275 s, 12.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0896366 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0849991 s, 12.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0888024 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0856621 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0899346 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0898194 s, 11.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0934757 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0926941 s, 11.3 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg305-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg305-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8443 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.896 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (126s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 20:02:09 (1781049729) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg305-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.221314 s, 75.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0353295 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0510909 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0455995 s, 23.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0566736 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0669229 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0648897 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0782618 s, 13.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0727709 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0890257 s, 11.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.095058 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0873988 s, 12.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0837399 s, 12.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0945242 s, 11.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0975577 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113023 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138756 s, 7.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193817 s, 54.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228303 s, 45.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244715 s, 42.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224832 s, 46.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224858 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320374 s, 32.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0150336 s, 69.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00618226 s, 170 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023711 s, 44.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00595848 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00400082 s, 262 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295678 s, 35.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0302549 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00366784 s, 286 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00572846 s, 183 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00432618 s, 242 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg305-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg305-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.02182 s, 20.9 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73424 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.5@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.5@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.229186 s, 73.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320879 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0620895 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0769897 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0758135 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102877 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112295 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12223 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139708 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132169 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125069 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115658 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12006 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.115384 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108654 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.194309 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.203111 s, 5.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201069 s, 52.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278226 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177272 s, 59.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231881 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199086 s, 52.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327079 s, 32.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255522 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312234 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871362 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596279 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00584519 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00552861 s, 190 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.004699 s, 223 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790474 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0047047 s, 223 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00333657 s, 314 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg305-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg305-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.9954 s, 21.0 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74088 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.5@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg305-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.211795 s, 79.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.037753 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0547637 s, 19.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0684436 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.065595 s, 16.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0663475 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.073093 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0865389 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0715687 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0904165 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0999115 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0885151 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100422 s, 10.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10412 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0934164 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0906187 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119183 s, 8.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864392 s, 121 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0135495 s, 77.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128589 s, 81.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152205 s, 68.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891799 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221198 s, 47.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136222 s, 77.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209331 s, 50.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022527 s, 46.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00528031 s, 199 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023195 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00389626 s, 269 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233626 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00386441 s, 271 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00594614 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00151704 s, 691 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg305-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg305-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.96594 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.89364 s, 21.4 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg305-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.356578 s, 47.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0697341 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139919 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121291 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103859 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140056 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130774 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120428 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118784 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134516 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139801 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154192 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138984 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132427 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168479 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.324572 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.331427 s, 3.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242811 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255116 s, 41.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291335 s, 36.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271051 s, 38.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896273 s, 117 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304636 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0320211 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246897 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118665 s, 88.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225953 s, 46.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198441 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211206 s, 49.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168018 s, 62.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00938244 s, 112 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085434 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126548 s, 82.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg305-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg305-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.78403 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74629 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.92838 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74708 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg305-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.671006 s, 25.0 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0662163 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0819932 s, 12.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0609662 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0679182 s, 15.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0711164 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0616985 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0837855 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0969725 s, 10.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.315561 s, 3.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.320984 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.324996 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317436 s, 3.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.329484 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.323075 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.354459 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.334671 s, 3.1 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128899 s, 81.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149567 s, 70.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188242 s, 55.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145367 s, 72.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160448 s, 65.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206613 s, 50.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01809 s, 58.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160998 s, 65.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200236 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00457146 s, 229 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694773 s, 151 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0094337 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00446708 s, 235 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103046 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0053008 s, 198 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0041211 s, 254 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg305-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg305-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.5018 s, 5.4 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4388 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg305-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.216708 s, 77.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299129 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0441978 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0608745 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0555158 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0744704 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0830161 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.05838 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0770685 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0974873 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0806951 s, 13.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0809303 s, 13.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0932501 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0972855 s, 10.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0675149 s, 15.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0899669 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103217 s, 10.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233169 s, 45.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257955 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269516 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136855 s, 76.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119089 s, 88.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031353 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028783 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268867 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244896 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105107 s, 99.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0137266 s, 76.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165465 s, 63.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274779 s, 38.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289984 s, 36.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00500976 s, 209 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162523 s, 64.5 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg305-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg305-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.97476 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74628 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg305-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.364785 s, 46.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0468887 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0350213 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0407023 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0543988 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0413493 s, 25.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0367691 s, 28.5 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0527815 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0484302 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154653 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.186326 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.199183 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.195855 s, 5.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.330499 s, 3.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.328115 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.328997 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.354061 s, 3.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145627 s, 72.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192535 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241685 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170831 s, 61.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237076 s, 44.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177231 s, 59.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210167 s, 49.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221152 s, 47.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250085 s, 41.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184691 s, 56.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236386 s, 44.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207472 s, 50.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225511 s, 46.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00417889 s, 251 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.004897 s, 214 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00328806 s, 319 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg305-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg305-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.7805 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74231 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.76785 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73061 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (362s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 20:08:11 (1781050091) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (4s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 20:08:15 (1781050095) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg305-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.48884 s, 301 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg305-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.0346 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (53s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 20:09:08 (1781050148) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.257546 s, 65.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0466817 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0509996 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0910286 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113396 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372522 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0436981 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.138739 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0617057 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0647923 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.283465 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.317425 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.293156 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.256537 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.260276 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.240462 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.349856 s, 3.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0720023 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0568973 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109257 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0505032 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0610294 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107459 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0932223 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0620801 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0776765 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0650599 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0858366 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0629778 s, 16.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0892731 s, 11.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0667135 s, 15.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0764927 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.07462 s, 14.1 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg305-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg305-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 8.19108 s, 12.8 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.31074 s, 19.7 MB/s Read runtime is 7 s, speed is 14.285714 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.690065 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131907 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150809 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.409418 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.467158 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.446415 s, 2.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.48566 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.467391 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.506304 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.549762 s, 1.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.529288 s, 2.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.484143 s, 2.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.550788 s, 1.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.497776 s, 2.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.519365 s, 2.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff975f46293000.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff975f46293000.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.545756 s, 1.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.636979 s, 1.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.551009 s, 1.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.530685 s, 2.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.524608 s, 2.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.540068 s, 1.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.543952 s, 1.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.587966 s, 1.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.557436 s, 1.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.581525 s, 1.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.527638 s, 2.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.568178 s, 1.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.578704 s, 1.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.601102 s, 1.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.597565 s, 1.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.596174 s, 1.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.604051 s, 1.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.60596 s, 1.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff975f46293000.lock_unused_count=3 removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg305-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg305-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 7.60234 s, 13.8 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.37675 s, 19.5 MB/s Read runtime is 7 s, speed is 14.285714 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (90s) == sanityn test 77o: Changing rank should not panic ====== 20:10:38 (1781050238) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (7s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 20:10:45 (1781050245) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (70s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 20:11:55 (1781050315) mds.MDS.mdt.nrs_policies=tbf oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg305-client: oleg305-server: ssh exited with exit code 36 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg305-client: oleg305-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (20s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 20:12:15 (1781050335) oleg305-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device pdsh@oleg305-client: oleg305-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (6s) == sanityn test 79: xattr: intent error ================== 20:12:21 (1781050341) File: /mnt/lustre/d79.sanityn Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505430 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-09 20:12:22.000000000 -0400 Modify: 2026-06-09 20:12:22.000000000 -0400 Change: 2026-06-09 20:12:22.000000000 -0400 Birth: 2026-06-09 20:12:22.000000000 -0400 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (15s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 20:12:36 (1781050356) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (2s) == sanityn test 80b: Accessing directory during migration ========================================================== 20:12:38 (1781050358) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (2s) == sanityn test 81a: rename and stat under striped directory ========================================================== 20:12:40 (1781050360) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (2s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 20:12:42 (1781050362) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (2s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 20:12:44 (1781050364) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (2s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 20:12:46 (1781050366) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7129 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7129 PASS 82 (4s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 20:12:50 (1781050370) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (2s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 20:12:52 (1781050372) fail_loc=0x8000060b PASS 84 (10s) == sanityn test 90: open/create and unlink striped directory ========================================================== 20:13:02 (1781050382) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (2s) == sanityn test 91: chmod and unlink striped directory === 20:13:04 (1781050384) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 20:13:05 (1781050385) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (2s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 20:13:07 (1781050387) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2627 0xa43 0 0 2647 0xa57 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2628 0xa44 0 0 2648 0xa58 0 rc1=0 and rc2=0 PASS 93 (12s) == sanityn test 94: signal vs CP callback race =========== 20:13:19 (1781050399) osc.lustre-OST0000-osc-ffff975f46293000.idle_timeout=0 osc.lustre-OST0000-osc-ffff975f48093800.idle_timeout=0 osc.lustre-OST0001-osc-ffff975f46293000.idle_timeout=0 osc.lustre-OST0001-osc-ffff975f48093800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.110157 s, 74.4 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 3.08332 s, 1.3 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 283429 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0607985 s, 67.4 kB/s osc.lustre-OST0000-osc-ffff975f46293000.idle_timeout=debug osc.lustre-OST0000-osc-ffff975f48093800.idle_timeout=debug osc.lustre-OST0001-osc-ffff975f46293000.idle_timeout=debug osc.lustre-OST0001-osc-ffff975f48093800.idle_timeout=debug PASS 94 (12s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 20:13:31 (1781050411) llite.lustre-ffff975f46293000.fast_read=0 llite.lustre-ffff975f48093800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.00411701 s, 3.0 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff975f46293000.fast_read=1 llite.lustre-ffff975f48093800.fast_read=1 PASS 95b (10s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 20:13:41 (1781050421) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (2s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 20:13:43 (1781050423) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00303445 s, 1.3 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (4s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 20:13:47 (1781050427) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0571258 s, 36.7 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (4s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 20:13:51 (1781050431) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0727603 s, 28.8 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (5s) == sanityn test 100e: DoM: read on open and file size ==== 20:13:56 (1781050436) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.00369479 s, 3.2 MB/s PASS 100e (4s) == sanityn test 101a: Discard DoM data on unlink ========= 20:14:00 (1781050440) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000694499 s, 5.9 MB/s PASS 101a (5s) == sanityn test 101b: Discard DoM data on rename ========= 20:14:05 (1781050445) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00300768 s, 1.4 MB/s PASS 101b (4s) == sanityn test 101c: Discard DoM data on close-unlink === 20:14:09 (1781050449) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0033268 s, 1.2 MB/s PASS 101c (5s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 20:14:15 (1781050455) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1781050457 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1781050460 Finishing test test23 at 1781050460 PASS 103 (10s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 20:14:25 (1781050465) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (2s) == sanityn test 105: Glimpse and lock cancel race ======== 20:14:27 (1781050467) fail_loc=0x416 PASS 105 (50s) == sanityn test 106a: Verify the btime via statx() ======= 20:15:17 (1781050517) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (2s) == sanityn test 106b: Glimpse RPCs test for statx ======== 20:15:19 (1781050519) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00551471 s, 186 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505458 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-09 20:15:20.000000000 -0400 Modify: 2026-06-09 20:15:20.000000000 -0400 Change: 2026-06-09 20:15:20.000000000 -0400 Birth: 2026-06-09 20:15:20.000000000 -0400 /mnt/lustre/f106b.sanityn 144115205272505458 -rw-r--r-- 1 0 0 1781050520 1781050520 1781050520 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505458 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-09 20:15:20.000000000 -0400 Modify: 2026-06-09 20:15:20.000000000 -0400 Change: 2026-06-09 20:15:20.000000000 -0400 Birth: 2026-06-09 20:15:20.000000000 -0400 1781050520 1024 1 PASS 106b (4s) == sanityn test 106c: Verify statx attributes mask ======= 20:15:23 (1781050523) PASS 106c (4s) == sanityn test 107a: Basic grouplock conflict =========== 20:15:27 (1781050527) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.133266 s, 78.7 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2634 0xa4a 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 First grouplock blocks second one PASS 107a (6s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 20:15:33 (1781050533) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.152347 s, 68.8 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2655 0xa5f 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7129 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (10s) == sanityn test 108a: lseek: parallel updates ============ 20:15:43 (1781050543) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.00428032 s, 2.3 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 4.57067 s, 1.8 MB/s PASS 108a (9s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 20:15:52 (1781050552) 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Iteration 7 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre 192.168.203.105@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.203.105@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg305-client.virtnet /mnt/lustre (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg305-server' oleg305-server: oleg305-server.virtnet: executing load_modules_local oleg305-server: Loading modules from /home/green/git/lustre-release/lustre oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg305-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg305-server: detected 4 online CPUs by sysfs oleg305-server: Force libcfs to create 2 CPU partitions Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre Starting client: oleg305-client.virtnet: -o user_xattr,flock oleg305-server@tcp:/lustre /mnt/lustre2 PASS 109 (647s) == sanityn test 110: do not grant another lock on resend ========================================================== 20:26:39 (1781051199) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-06-09 20:26:40.000000000 -0400 Modify: 2026-06-09 20:26:40.000000000 -0400 Change: 2026-06-09 20:26:40.000000000 -0400 Birth: 2026-06-09 20:26:40.000000000 -0400 fail_loc=0 fail_val=0 Connected clients: oleg305-client.virtnet oleg305-client.virtnet PASS 110 (37s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 20:27:16 (1781051236) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 20:27:17 (1781051237) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 20:27:18 (1781051238) lustre-OST0000-osc-ffff975f62269800 active. lustre-OST0001-osc-ffff975f62269800 active. lustre-OST0000-osc-ffff975f62269800 active. lustre-OST0001-osc-ffff975f62269800 active. MGC192.168.203.105@tcp active. PASS 113 (3s) cleanup: ====================================================== == sanityn test complete, duration 4824 sec ============== 20:27:21 (1781051241) Stopping clients: oleg305-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg305-client.virtnet /mnt/lustre2 opts: