-----============= acceptance-small: sanityn ============----- Sat Feb 28 13:53:14 EST 2026 oleg229-server: ls: cannot access '/home/green/git/lustre-release/lustre/tests/except/sanityn.*ex': No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Started clients oleg229-client.virtnet: 192.168.202.129@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt) oleg229-client.virtnet: executing check_config_client /mnt/lustre oleg229-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg229-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8d6402f5b800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8d6418e92800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8d6402f5b800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8d6418e92800.idle_timeout=debug disable quota as required oleg229-server: oleg229-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167561 s, 62.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f7142] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 13:53:27 (1772304807) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (2s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 13:53:30 (1772304810) -rw-r--r-- 1 root root 0 Feb 28 13:53 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (1s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 13:53:32 (1772304812) -rw-r--r-- 1 root root 0 Feb 28 13:53 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (2s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 13:53:34 (1772304814) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (1s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 13:53:35 (1772304815) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (2s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 13:53:37 (1772304817) total 1 -rwxrwxrwx 1 root root 0 Feb 28 13:53 f2a -rwxrwxrwx 1 root root 0 Feb 28 13:53 f2b total 1 -rwxrwxrwx 1 root root 0 Feb 28 13:53 f2a -rwxrwxrwx 1 root root 0 Feb 28 13:53 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes copied, 0.00243405 s, 210 kB/s PASS 2e (2s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 13:53:39 (1772304819) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 13:53:40 (1772304820) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0820482 s, 25.6 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (2s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 13:53:42 (1772304822) PASS 3 (2s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 13:53:44 (1772304824) PASS 4 (3s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 13:53:47 (1772304827) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (2s) == sanityn test 6: remove of open file on other node ============================================================================ 13:53:49 (1772304829) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (2s) == sanityn test 7: remove of open directory on other node ======================================================================= 13:53:51 (1772304831) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (2s) == sanityn test 8: remove of open special file on other node ==================================================================== 13:53:53 (1772304833) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (2s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 13:53:55 (1772304835) PASS 9a (2s) == sanityn test 9b: append to striped sparse file ======== 13:53:57 (1772304837) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133068 s, 78.8 MB/s 3+0 records in 3+0 records out 3 bytes copied, 0.00948839 s, 0.3 kB/s Data read (expecting 'foo'): foo PASS 9b (2s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 13:53:59 (1772304839) 1+0 records in 1+0 records out 1 byte copied, 0.00133827 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00161492 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00169043 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0016188 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00145803 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00163579 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.0016231 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00142 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00156759 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00146324 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00148606 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte copied, 0.00146215 s, 0.7 kB/s PASS 10a (3s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 13:54:02 (1772304842) 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.00311466 s, 986 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00678534 s, 604 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB, 3.0 KiB) copied, 0.000313369 s, 9.8 MB/s PASS 10b (2s) == sanityn test 11: execution of file opened for write should return error ============================================================== 13:54:04 (1772304844) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (2s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 13:54:06 (1772304846) warning: '-runtime' deprecated, use '-t runtime' instead - link 2377 (time 1772304856.92 total 10.00 last 237.65) - link 4510 (time 1772304866.92 total 20.00 last 213.24) - link 6649 (time 1772304876.92 total 30.00 last 213.90) - link 8949 (time 1772304886.92 total 40.01 last 229.94) - link 10000 (time 1772304892.02 total 45.10 last 206.43) - link 12253 (time 1772304902.02 total 55.10 last 225.20) - link 14389 (time 1772304912.02 total 65.11 last 213.50) - link 16575 (time 1772304922.03 total 75.11 last 218.51) - link 18664 (time 1772304932.03 total 85.11 last 208.87) - link 20000 (time 1772304937.64 total 90.72 last 238.31) total: 22023 link in 100.00 seconds: 220.23 ops/second using seed 855803961 running for 100 seconds - stat 10000 (time 1772304852 ; total 5 ; last 5) - stat 20000 (time 1772304856 ; total 9 ; last 4) - stat 30000 (time 1772304861 ; total 14 ; last 5) - stat 40000 (time 1772304865 ; total 18 ; last 4) - stat 50000 (time 1772304870 ; total 23 ; last 5) - stat 60000 (time 1772304875 ; total 28 ; last 5) - stat 70000 (time 1772304879 ; total 32 ; last 4) - stat 80000 (time 1772304883 ; total 36 ; last 4) - stat 90000 (time 1772304888 ; total 41 ; last 5) - stat 100000 (time 1772304892 ; total 45 ; last 4) - stat 110000 (time 1772304897 ; total 50 ; last 5) - stat 120000 (time 1772304906 ; total 59 ; last 9) - stat 129764 (time 1772304917 ; total 70 ; last 11) - stat 130000 (time 1772304917 ; total 70 ; last 0) - stat 139208 (time 1772304928 ; total 81 ; last 11) - stat 140000 (time 1772304928 ; total 81 ; last 0) - stat 149387 (time 1772304939 ; total 92 ; last 11) - stat 150000 (time 1772304939 ; total 92 ; last 0) total: 156651 stats in 100 seconds: 1566.510010 stats/second - unlinked 0 (time 1772304897 ; total 0 ; last 0) - unlinked 10000 (time 1772304954 ; total 57 ; last 57) - unlinked 20000 (time 1772304981 ; total 84 ; last 27) unlink(/mnt/lustre2/lockdir/lockfile22023) error: No such file or directory total: 22022 unlinks in 89 seconds: 247.438202 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (19691) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (19693) - No such process PASS 12 (141s) == sanityn test 13: test directory page revocation ======= 13:56:27 (1772304987) f13 f13 PASS 13 (3s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 13:56:30 (1772304990) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 20862 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 13:56:32 (1772304992) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 21434 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (2s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 13:56:34 (1772304994) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 22006 Terminated $DIR1/$tdir/sleep 60 PASS 14b (2s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 13:56:36 (1772304996) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 22581 Terminated $DIR1/$tdir/sleep 60 PASS 14c (2s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 13:56:39 (1772304999) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 23156 Terminated $DIR1/$tdir/sleep 60 PASS 14d (2s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 13:56:41 (1772305001) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7519232 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7519232kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (12s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 13:56:53 (1772305013) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0774704 s, 27.1 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0637691 s, 32.9 MB/s Chance of close/open is 1 in 50 Seed set to 5014 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[0] 1772305014.506246 write 0x1901f9 thru 0x19fc45 (0xfa4d bytes) 000003[1] 1772305014.516345 read 0x0dea9e thru 0x0e4af0 (0x6053 bytes) 000004[0] 1772305014.528944 read 0x02a5dc thru 0x03538f (0xadb4 bytes) 000005[0] 1772305014.533346 mapread 0x1143ec thru 0x123df7 (0xfa0c bytes) 000006[0] 1772305014.539715 mapwrite 0x5540ce thru 0x55b5a4 (0x74d7 bytes) 000009[1] 1772305014.588997 mapread 0x462c18 thru 0x47006e (0xd457 bytes) 000011[0] 1772305014.606537 mapwrite 0x33003f thru 0x33c432 (0xc3f4 bytes) truncating to largest ever: 0xe8737 000014[1] 1772305014.636008 trunc from 0x55b5a5 to 0x0e8736 (0x472e6e bytes) 000015[0] 1772305014.651253 write 0x70749b thru 0x71564b (0xe1b1 bytes) 000016[1] 1772305014.660967 read 0x6b42fd thru 0x6c07b6 (0xc4ba bytes) 000017[1] 1772305014.681584 read 0x0ac243 thru 0x0b710d (0xaecb bytes) 000020[1] 1772305014.698450 read 0x07da55 thru 0x07e077 (0x623 bytes) 000022[1] 1772305014.710308 mapwrite 0x6b3363 thru 0x6b80b9 (0x4d57 bytes) 000024[0] 1772305014.753605 write 0x2f8a66 thru 0x305cdc (0xd277 bytes) 000025[0] 1772305014.761421 mapread 0x43fe78 thru 0x44c98b (0xcb14 bytes) 000026[1] 1772305014.773307 read 0x31ac6a thru 0x31f0fd (0x4494 bytes) 000030[0] 1772305014.790123 mapread 0x533cb6 thru 0x53c73c (0x8a87 bytes) 000031[0] 1772305014.794751 read 0x2647a2 thru 0x271d37 (0xd596 bytes) 000033[0] 1772305014.820194 mapwrite 0x5f58a1 thru 0x5fecc7 (0x9427 bytes) 000037[0] 1772305014.892372 mapwrite 0x45e11e thru 0x45fa88 (0x196b bytes) 000038[1] 1772305014.908587 read 0x484915 thru 0x49439e (0xfa8a bytes) 000046[0] 1772305014.940212 read 0x25650b thru 0x25c8d6 (0x63cc bytes) 000047[0] 1772305014.982916 mapwrite 0x2bb40e thru 0x2c06da (0x52cd bytes) 000048[1] 1772305015.031118 mapwrite 0x6342f9 thru 0x63635d (0x2065 bytes) truncating to largest ever: 0x8f54b6 000052[0] 1772305015.068676 trunc from 0x71564c to 0x8f54b5 (0x1dfe6a bytes) 000053[0] 1772305015.080441 mapwrite 0x1c6554 thru 0x1d4099 (0xdb46 bytes) 000059[1] 1772305015.150510 write 0x035740 thru 0x03c359 (0x6c1a bytes) 000060[1] 1772305015.159830 trunc from 0x8f54b6 to 0x7e0025 (0x115490 bytes) 000062[0] 1772305015.179846 mapread 0x5e3005 thru 0x5e5519 (0x2515 bytes) 000064[1] 1772305015.195744 mapread 0x70452e thru 0x70b6b6 (0x7189 bytes) 000066[0] 1772305015.236099 trunc from 0x7e0026 to 0x6b788a (0x12879b bytes) 000067[1] 1772305015.254321 read 0x17dafc thru 0x187ee2 (0xa3e7 bytes) 000069[0] 1772305015.268188 write 0x5c36b8 thru 0x5ca65e (0x6fa7 bytes) 000072[1] 1772305015.278592 trunc from 0x6b788b to 0x0146c6 (0x6a31c4 bytes) 000073[1] 1772305015.293706 mapread 0x001e01 thru 0x0034a6 (0x16a6 bytes) 000075[0] 1772305015.303246 trunc from 0x0146c7 to 0x0833ea (0x6ed24 bytes) 000078[1] 1772305015.312034 write 0x5f287f thru 0x6027c5 (0xff47 bytes) 000079[1] 1772305015.319598 write 0x5ba521 thru 0x5c9526 (0xf006 bytes) 000081[1] 1772305015.360577 mapwrite 0x10f7d7 thru 0x11bc0a (0xc434 bytes) 000083[1] 1772305015.396903 read 0x2d868f thru 0x2e3633 (0xafa5 bytes) 000086[1] 1772305015.402740 write 0x226e91 thru 0x227edf (0x104f bytes) 000088[1] 1772305015.411093 trunc from 0x6027c6 to 0x5143e0 (0xee3e5 bytes) 000089[1] 1772305015.417532 mapwrite 0x7bac68 thru 0x7cac2d (0xffc6 bytes) 000090[0] 1772305015.463461 write 0x747874 thru 0x74d3c6 (0x5b53 bytes) 000091[0] 1772305015.473365 write 0x9f1c52 thru 0x9f97f9 (0x7ba8 bytes) 000096[1] 1772305015.493256 mapread 0x348dec thru 0x351cb0 (0x8ec5 bytes) 000099[0] 1772305015.522095 write 0x877052 thru 0x877be6 (0xb95 bytes) 000100[1] 1772305015.531753 mapwrite 0x3026cd thru 0x311204 (0xeb38 bytes) 000101[1] 1772305015.579399 write 0x2e2901 thru 0x2e9caa (0x73aa bytes) 000104[1] 1772305015.590647 mapwrite 0x144ce2 thru 0x15032f (0xb64e bytes) truncating to largest ever: 0x9c2952 000105[0] 1772305015.620838 trunc from 0x9f97fa to 0x9c2951 (0x36ea8 bytes) 000113[1] 1772305015.651445 read 0x51829d thru 0x5275a4 (0xf308 bytes) 000114[1] 1772305015.665376 mapwrite 0x469b86 thru 0x46ee22 (0x529d bytes) 000115[1] 1772305015.731274 mapwrite 0x5e5602 thru 0x5e72f8 (0x1cf7 bytes) 000116[1] 1772305015.766743 mapwrite 0x412d05 thru 0x415a01 (0x2cfd bytes) 000118[1] 1772305015.790990 mapwrite 0x166c59 thru 0x175e2a (0xf1d2 bytes) 000123[1] 1772305015.838245 mapwrite 0x4f4c34 thru 0x4fe96d (0x9d3a bytes) 000125[0] 1772305015.872737 trunc from 0x9c2952 to 0x5178b4 (0x4ab09d bytes) 000126[1] 1772305015.888451 write 0x4fc76b thru 0x50bbda (0xf470 bytes) 000128[0] 1772305015.906172 trunc from 0x5178b5 to 0x277662 (0x2a0252 bytes) 000129[0] 1772305015.920557 read 0x106928 thru 0x106dac (0x485 bytes) 000131[1] 1772305015.939060 mapwrite 0x55d9f9 thru 0x563f26 (0x652e bytes) 000133[1] 1772305015.980760 mapread 0x11f123 thru 0x120224 (0x1102 bytes) 000134[0] 1772305015.983710 mapwrite 0x14a029 thru 0x1598c2 (0xf89a bytes) 000138[1] 1772305016.039811 mapwrite 0x3de6db thru 0x3ed89e (0xf1c4 bytes) 000140[0] 1772305016.088178 read 0x254d71 thru 0x25c411 (0x76a1 bytes) 000144[1] 1772305016.102991 mapwrite 0x9f0588 thru 0x9ff5b4 (0xf02d bytes) 000145[1] 1772305016.158256 trunc from 0x9ff5b5 to 0x4ff051 (0x500563 bytes) 000146[1] 1772305016.169077 write 0x147300 thru 0x14dbbc (0x68bd bytes) 000147[0] 1772305016.173306 mapwrite 0x591f3e thru 0x597333 (0x53f6 bytes) 000152[1] 1772305016.226733 write 0x1ffbdc thru 0x204a1a (0x4e3f bytes) 000153[1] 1772305016.242226 write 0x24fed6 thru 0x253024 (0x314f bytes) 000154[1] 1772305016.246338 trunc from 0x597334 to 0x161628 (0x435d0b bytes) 000156[1] 1772305016.257370 mapread 0x09d102 thru 0x0a311b (0x601a bytes) 000158[0] 1772305016.269478 read 0x0ca4bc thru 0x0d5aab (0xb5f0 bytes) 000159[0] 1772305016.276547 read 0x06e0c4 thru 0x0793cd (0xb30a bytes) 000161[1] 1772305016.288811 trunc from 0x161629 to 0x3b2c4f (0x251627 bytes) 000164[0] 1772305016.308203 mapwrite 0x21c317 thru 0x21f96f (0x3659 bytes) 000166[1] 1772305016.337853 read 0x2be030 thru 0x2c530e (0x72df bytes) 000167[1] 1772305016.347338 write 0x02d786 thru 0x033329 (0x5ba4 bytes) 000168[1] 1772305016.356352 mapread 0x3232b4 thru 0x332d34 (0xfa81 bytes) 000169[0] 1772305016.361135 write 0x1c1a65 thru 0x1c389e (0x1e3a bytes) 000172[1] 1772305016.383856 mapwrite 0x1d4fc7 thru 0x1d5cc5 (0xcff bytes) 000174[1] 1772305016.414965 read 0x317559 thru 0x31c823 (0x52cb bytes) 000175[1] 1772305016.421201 write 0x92113b thru 0x928956 (0x781c bytes) 000176[1] 1772305016.428324 mapwrite 0x4a6ea8 thru 0x4b0f33 (0xa08c bytes) 000178[0] 1772305016.490223 write 0x78fe86 thru 0x79ec7a (0xedf5 bytes) 000179[0] 1772305016.505607 mapread 0x27c96e thru 0x2828a5 (0x5f38 bytes) 000181[0] 1772305016.525276 mapread 0x4d3496 thru 0x4d7409 (0x3f74 bytes) 000182[0] 1772305016.533898 mapread 0x35a01e thru 0x35be7d (0x1e60 bytes) 000184[1] 1772305016.537045 mapwrite 0x38fbb0 thru 0x396e39 (0x728a bytes) 000187[1] 1772305016.575023 mapread 0x44ae4c thru 0x455c60 (0xae15 bytes) 000188[0] 1772305016.590230 read 0x61ca6c thru 0x621dae (0x5343 bytes) 000189[1] 1772305016.601274 mapread 0x454627 thru 0x459508 (0x4ee2 bytes) 000192[0] 1772305016.616488 mapwrite 0x0d260e thru 0x0e0fc7 (0xe9ba bytes) 000193[0] 1772305016.645755 trunc from 0x928957 to 0x4a81e4 (0x480772 bytes) 000194[0] 1772305016.656966 trunc from 0x4a81e5 to 0x41e4e2 (0x89d02 bytes) 000195[1] 1772305016.664982 write 0x5ea2ec thru 0x5f8570 (0xe285 bytes) 000198[1] 1772305016.679942 mapread 0x0fafdd thru 0x10aa34 (0xfa58 bytes) 000201[1] 1772305016.706956 trunc from 0x5f8571 to 0x159182 (0x49f3ee bytes) 000202[0] 1772305016.714269 write 0x0fa9df thru 0x101bd2 (0x71f4 bytes) 000203[1] 1772305016.727061 read 0x151f79 thru 0x159182 (0x720a bytes) 000204[0] 1772305016.736041 mapread 0x151428 thru 0x15402a (0x2c03 bytes) 000207[0] 1772305016.748184 trunc from 0x159183 to 0x468d61 (0x30fbdf bytes) 000208[0] 1772305016.760469 read 0x2b676b thru 0x2b7b49 (0x13df bytes) 000210[1] 1772305016.783072 trunc from 0x468d62 to 0x42dce1 (0x3b080 bytes) 000213[0] 1772305016.801372 write 0x43556b thru 0x43ba19 (0x64af bytes) 000214[1] 1772305016.807719 mapread 0x3d98ca thru 0x3e92dd (0xfa14 bytes) 000215[0] 1772305016.840969 mapwrite 0x9b06a7 thru 0x9bb522 (0xae7c bytes) 000216[1] 1772305016.903682 mapread 0x43d8e5 thru 0x44663f (0x8d5b bytes) 000217[0] 1772305016.936509 read 0x70545f thru 0x70c811 (0x73b3 bytes) 000220[1] 1772305016.947750 write 0x58de02 thru 0x59c51a (0xe719 bytes) 000221[0] 1772305017.015440 write 0x963385 thru 0x96ea38 (0xb6b4 bytes) 000223[0] 1772305017.026619 mapwrite 0x479fa8 thru 0x4893f5 (0xf44e bytes) 000224[1] 1772305017.079752 mapwrite 0x1e43fe thru 0x1eb452 (0x7055 bytes) 000227[0] 1772305017.123942 trunc from 0x9bb523 to 0x0d56ba (0x8e5e68 bytes) 000228[1] 1772305017.133592 read 0x020eca thru 0x024bfa (0x3d31 bytes) 000229[1] 1772305017.143495 read 0x0b3e51 thru 0x0b7986 (0x3b36 bytes) 000230[1] 1772305017.149209 mapread 0x0acbd0 thru 0x0bc039 (0xf46a bytes) 000231[0] 1772305017.155206 mapwrite 0x9318e7 thru 0x93b7a3 (0x9ebd bytes) 000233[1] 1772305017.200154 mapwrite 0x5d3858 thru 0x5de4fc (0xaca5 bytes) 000237[0] 1772305017.253700 write 0x48ec20 thru 0x495f92 (0x7373 bytes) 000239[0] 1772305017.264640 trunc from 0x93b7a4 to 0x77ed5f (0x1bca44 bytes) 000240[0] 1772305017.276442 read 0x5d64e6 thru 0x5e47a7 (0xe2c2 bytes) 000241[1] 1772305017.291399 mapwrite 0x711fec thru 0x714aa8 (0x2abd bytes) 000242[0] 1772305017.320058 mapwrite 0x89f700 thru 0x8a707e (0x797f bytes) 000244[1] 1772305017.364197 mapread 0x791f63 thru 0x79c581 (0xa61f bytes) 000245[0] 1772305017.384789 read 0x6d6742 thru 0x6db01d (0x48dc bytes) 000246[0] 1772305017.385228 read 0x28f421 thru 0x296193 (0x6d73 bytes) 000247[1] 1772305017.426959 read 0x4d7461 thru 0x4e23a7 (0xaf47 bytes) 000248[0] 1772305017.470938 mapwrite 0x05c4b7 thru 0x060b2f (0x4679 bytes) 000251[0] 1772305017.501514 read 0x1dbb9e thru 0x1e34e4 (0x7947 bytes) 000252[0] 1772305017.524363 mapread 0x73ed7e thru 0x7402a3 (0x1526 bytes) 000255[1] 1772305017.533067 mapread 0x2cdf15 thru 0x2d13d4 (0x34c0 bytes) 000256[1] 1772305017.548411 trunc from 0x8a707f to 0x829e2c (0x7d252 bytes) 000258[1] 1772305017.571783 mapread 0x2214f4 thru 0x22ffa1 (0xeaae bytes) 000259[0] 1772305017.595692 read 0x6af215 thru 0x6bb378 (0xc164 bytes) 000261[1] 1772305017.607344 mapwrite 0x17e243 thru 0x18848f (0xa24d bytes) 000265[0] 1772305017.645467 trunc from 0x829e2d to 0x527ab0 (0x30237c bytes) 000266[0] 1772305017.656372 mapwrite 0x8072b2 thru 0x8084c9 (0x1218 bytes) 000267[1] 1772305017.684275 mapread 0x3dc035 thru 0x3de820 (0x27ec bytes) 000268[0] 1772305017.698625 mapwrite 0x6f5e0f thru 0x6f682d (0xa1f bytes) 000269[1] 1772305017.719549 read 0x57b676 thru 0x589636 (0xdfc1 bytes) 000272[1] 1772305017.730273 read 0x2258cf thru 0x23426c (0xe99e bytes) 000273[0] 1772305017.773762 trunc from 0x8084ca to 0x2179b5 (0x5f0b14 bytes) 000275[0] 1772305017.796666 read 0x2074ea thru 0x20e60e (0x7125 bytes) 000279[1] 1772305017.814137 mapwrite 0x478ba0 thru 0x47f8b7 (0x6d18 bytes) 000280[0] 1772305017.855446 write 0x66115c thru 0x66162d (0x4d2 bytes) 000281[0] 1772305017.863788 mapwrite 0x5c9b6b thru 0x5cb877 (0x1d0d bytes) 000282[1] 1772305017.940098 trunc from 0x66162e to 0x16fb98 (0x4f1a95 bytes) 000283[1] 1772305017.960620 read 0x012efc thru 0x01fe6d (0xcf72 bytes) 000286[0] 1772305017.980186 write 0x126d77 thru 0x12fe51 (0x90db bytes) 000287[1] 1772305017.990252 mapread 0x07b4fd thru 0x08048e (0x4f92 bytes) 000288[1] 1772305017.996187 write 0x3a983a thru 0x3b97d8 (0xff9f bytes) 000289[1] 1772305018.008250 mapwrite 0x03ceba thru 0x0491bd (0xc304 bytes) 000290[0] 1772305018.025505 read 0x0c2540 thru 0x0d0563 (0xe024 bytes) 000291[1] 1772305018.037164 mapwrite 0x4b493d thru 0x4bd077 (0x873b bytes) 000295[0] 1772305018.086577 mapread 0x383623 thru 0x383f98 (0x976 bytes) 000296[0] 1772305018.106783 mapread 0x2a9d25 thru 0x2b3751 (0x9a2d bytes) 000299[1] 1772305018.144274 read 0x020f25 thru 0x024534 (0x3610 bytes) 000300[0] 1772305018.185851 mapwrite 0x80720c thru 0x81629d (0xf092 bytes) 000301[0] 1772305018.266754 mapread 0x0fec8a thru 0x103dfc (0x5173 bytes) 000302[0] 1772305018.277342 write 0x702879 thru 0x71024a (0xd9d2 bytes) 000305[1] 1772305018.288068 write 0x0c4d15 thru 0x0d2638 (0xd924 bytes) 000307[1] 1772305018.307447 read 0x0f4171 thru 0x0f8f51 (0x4de1 bytes) 000308[0] 1772305018.314656 mapread 0x25c047 thru 0x26617d (0xa137 bytes) 000309[1] 1772305018.339653 read 0x560fd4 thru 0x5623c9 (0x13f6 bytes) 000310[0] 1772305018.354475 trunc from 0x81629e to 0x938183 (0x121ee6 bytes) 000311[1] 1772305018.372281 mapread 0x0e85c8 thru 0x0efcb0 (0x76e9 bytes) 000314[1] 1772305018.413187 write 0x802e41 thru 0x80f759 (0xc919 bytes) 000317[0] 1772305018.434768 trunc from 0x938184 to 0x5bbf82 (0x37c201 bytes) 000318[0] 1772305018.446572 mapwrite 0x21f24b thru 0x22cfdd (0xdd93 bytes) 000319[1] 1772305018.495326 read 0x108fa3 thru 0x109b01 (0xb5f bytes) 000321[0] 1772305018.514282 mapread 0x3588e4 thru 0x35b24a (0x2967 bytes) 000324[0] 1772305018.528681 trunc from 0x5bbf83 to 0x38b622 (0x230960 bytes) 000326[0] 1772305018.546419 mapwrite 0x860dbf thru 0x86d5e2 (0xc824 bytes) 000327[1] 1772305018.587777 write 0x3737cf thru 0x37b7a3 (0x7fd5 bytes) 000328[1] 1772305018.598232 trunc from 0x86d5e3 to 0x098637 (0x7d4fab bytes) 000332[0] 1772305018.625031 mapwrite 0x770896 thru 0x77bd87 (0xb4f2 bytes) 000336[1] 1772305018.675424 mapread 0x4ea377 thru 0x4ef4a4 (0x512e bytes) 000337[0] 1772305018.687552 mapread 0x34708d thru 0x356198 (0xf10c bytes) 000339[1] 1772305018.710803 trunc from 0x77bd88 to 0x103d6d (0x67801a bytes) 000340[1] 1772305018.725302 mapwrite 0x69fa90 thru 0x6a236d (0x28de bytes) 000342[1] 1772305018.757672 mapwrite 0x39914f thru 0x39b935 (0x27e7 bytes) 000343[1] 1772305018.776746 read 0x389ef8 thru 0x394482 (0xa58b bytes) 000344[1] 1772305018.784044 write 0x232d06 thru 0x2386ee (0x59e9 bytes) 000346[1] 1772305018.794737 mapwrite 0x7dbeaa thru 0x7e313c (0x7293 bytes) 000347[0] 1772305018.823172 mapwrite 0x87799f thru 0x881999 (0x9ffb bytes) 000348[0] 1772305018.861884 mapread 0x76e7f4 thru 0x776f07 (0x8714 bytes) 000350[1] 1772305018.873049 mapread 0x264418 thru 0x26dfcd (0x9bb6 bytes) 000351[1] 1772305018.886886 mapread 0x2c41aa thru 0x2c7aef (0x3946 bytes) 000352[0] 1772305018.892357 mapread 0x7c9477 thru 0x7cd51d (0x40a7 bytes) 000353[1] 1772305018.902217 mapwrite 0x91092b thru 0x91f72b (0xee01 bytes) 000357[1] 1772305018.948261 mapwrite 0x47a969 thru 0x489090 (0xe728 bytes) 000359[1] 1772305018.975192 read 0x4072fa thru 0x407394 (0x9b bytes) 000360[0] 1772305019.004543 trunc from 0x91f72c to 0x918f26 (0x6805 bytes) 000361[0] 1772305019.023233 mapwrite 0x4ab7db thru 0x4aeaaf (0x32d5 bytes) 000362[0] 1772305019.042506 mapread 0x108914 thru 0x112abe (0xa1ab bytes) 000363[0] 1772305019.060359 mapwrite 0x082d9a thru 0x08b8df (0x8b46 bytes) 000364[0] 1772305019.089464 write 0x7527ee thru 0x758876 (0x6089 bytes) 000365[1] 1772305019.092717 mapread 0x51c21e thru 0x5210eb (0x4ece bytes) 000367[1] 1772305019.112789 trunc from 0x918f27 to 0x8a937b (0x6fbab bytes) 000370[1] 1772305019.130998 read 0x78fe77 thru 0x79dbc9 (0xdd53 bytes) 000375[1] 1772305019.151938 mapwrite 0x4b0cdb thru 0x4be284 (0xd5aa bytes) 000376[1] 1772305019.185779 trunc from 0x8a937c to 0x6e2a22 (0x1c6959 bytes) 000377[1] 1772305019.192074 mapwrite 0x8eb25e thru 0x8f156a (0x630d bytes) 000379[1] 1772305019.215160 mapwrite 0x4a1cb2 thru 0x4b104f (0xf39e bytes) 000380[1] 1772305019.252128 mapread 0x30cadf thru 0x31bf95 (0xf4b7 bytes) 000382[1] 1772305019.274673 trunc from 0x8f156b to 0x2994aa (0x6580c0 bytes) 000384[1] 1772305019.285704 write 0x27d5bb thru 0x27d9f3 (0x439 bytes) 000386[1] 1772305019.287614 mapwrite 0x8e20c0 thru 0x8e4205 (0x2146 bytes) 000389[1] 1772305019.315337 mapread 0x7075d9 thru 0x71757c (0xffa4 bytes) 000390[0] 1772305019.334629 read 0x0f752e thru 0x106e63 (0xf936 bytes) 000391[0] 1772305019.375403 mapread 0x37bb58 thru 0x380e89 (0x5332 bytes) 000393[1] 1772305019.382002 read 0x7f2f8a thru 0x7f32ab (0x322 bytes) 000394[1] 1772305019.389004 mapread 0x38874a thru 0x394429 (0xbce0 bytes) 000395[0] 1772305019.404429 write 0x3eebf3 thru 0x3f5d04 (0x7112 bytes) 000397[0] 1772305019.416969 mapread 0x4b8e6d thru 0x4c4c86 (0xbe1a bytes) 000399[0] 1772305019.420724 mapwrite 0x5f4bfa thru 0x601880 (0xcc87 bytes) 000401[0] 1772305019.481636 read 0x504fb5 thru 0x507f48 (0x2f94 bytes) 000404[0] 1772305019.515254 read 0x69618c thru 0x69ba85 (0x58fa bytes) 000405[1] 1772305019.519245 read 0x78d056 thru 0x794828 (0x77d3 bytes) 000406[1] 1772305019.531270 write 0x31e4f8 thru 0x32939c (0xaea5 bytes) 000407[1] 1772305019.540031 write 0x6fbdcd thru 0x70301b (0x724f bytes) 000411[0] 1772305019.566533 write 0x9e90e0 thru 0x9f021b (0x713c bytes) 000413[0] 1772305019.579692 trunc from 0x9f021c to 0x465c77 (0x58a5a4 bytes) 000420[1] 1772305019.603159 read 0x1745f0 thru 0x17487f (0x290 bytes) 000421[0] 1772305019.612233 mapread 0x2a0d31 thru 0x2a5205 (0x44d5 bytes) 000422[0] 1772305019.619313 trunc from 0x465c78 to 0x78e2ca (0x328653 bytes) 000423[0] 1772305019.631717 mapwrite 0x6495c1 thru 0x65500d (0xba4d bytes) 000424[1] 1772305019.665298 write 0x8a1e2e thru 0x8a42ef (0x24c2 bytes) 000425[0] 1772305019.672853 write 0x2d65cc thru 0x2e11e0 (0xac15 bytes) 000426[0] 1772305019.680042 mapwrite 0x6d314b thru 0x6e01ef (0xd0a5 bytes) 000428[1] 1772305019.711812 mapwrite 0x1b9b79 thru 0x1c93ef (0xf877 bytes) 000430[1] 1772305019.759835 mapread 0x315e20 thru 0x31fa61 (0x9c42 bytes) 000431[0] 1772305019.776760 write 0x865060 thru 0x87110b (0xc0ac bytes) 000433[0] 1772305019.786890 read 0x62d52b thru 0x634f89 (0x7a5f bytes) 000434[0] 1772305019.797901 read 0x600ec3 thru 0x606306 (0x5444 bytes) 000435[1] 1772305019.804770 mapread 0x6cdb73 thru 0x6dacd6 (0xd164 bytes) 000436[1] 1772305019.856647 write 0x8ca881 thru 0x8cbbf2 (0x1372 bytes) 000438[1] 1772305019.869010 read 0x656ab7 thru 0x65eed0 (0x841a bytes) 000440[1] 1772305019.886167 mapwrite 0x782b89 thru 0x786947 (0x3dbf bytes) 000441[1] 1772305019.902669 write 0x995a3d thru 0x99fe35 (0xa3f9 bytes) 000442[0] 1772305019.903647 read 0x573987 thru 0x5776d0 (0x3d4a bytes) 000443[0] 1772305019.916397 write 0x275714 thru 0x27f1ef (0x9adc bytes) 000444[1] 1772305019.951947 mapwrite 0x37aaab thru 0x386bf8 (0xc14e bytes) 000445[1] 1772305019.998190 mapread 0x3b7f8a thru 0x3bf86d (0x78e4 bytes) 000446[1] 1772305020.013266 read 0x2f5ff1 thru 0x304787 (0xe797 bytes) 000448[1] 1772305020.078716 trunc from 0x99fe36 to 0x757e69 (0x247fcc bytes) 000449[0] 1772305020.094681 mapread 0x234be6 thru 0x237dac (0x31c7 bytes) 000451[1] 1772305020.106619 mapwrite 0x5e31e8 thru 0x5f150c (0xe325 bytes) 000452[0] 1772305020.129228 mapread 0x0861c9 thru 0x08bb98 (0x59d0 bytes) 000453[0] 1772305020.142405 mapwrite 0x3c5baf thru 0x3ce1df (0x8631 bytes) 000455[1] 1772305020.176237 write 0x389640 thru 0x390bd1 (0x7592 bytes) 000459[1] 1772305020.192800 write 0x3e8531 thru 0x3f027d (0x7d4d bytes) 000460[1] 1772305020.195450 mapwrite 0x6f887d thru 0x7041c7 (0xb94b bytes) 000462[0] 1772305020.247327 read 0x30e356 thru 0x31b161 (0xce0c bytes) 000464[1] 1772305020.262321 read 0x363a7c thru 0x364128 (0x6ad bytes) 000465[0] 1772305020.307050 read 0x28db6e thru 0x2937b1 (0x5c44 bytes) 000467[0] 1772305020.352981 mapread 0x207767 thru 0x20c639 (0x4ed3 bytes) 000468[0] 1772305020.360930 write 0x46c780 thru 0x47340f (0x6c90 bytes) 000471[1] 1772305020.377511 mapwrite 0x82af8d thru 0x831e64 (0x6ed8 bytes) 000472[0] 1772305020.429328 read 0x536131 thru 0x53abf4 (0x4ac4 bytes) 000473[1] 1772305020.443618 mapwrite 0x8ddee8 thru 0x8e20ce (0x41e7 bytes) 000474[1] 1772305020.479933 mapread 0x57aa94 thru 0x584e6b (0xa3d8 bytes) 000479[1] 1772305020.504374 mapwrite 0x0aa8a1 thru 0x0b7d5b (0xd4bb bytes) 000482[1] 1772305020.548995 mapwrite 0x8b1cc3 thru 0x8bf614 (0xd952 bytes) 000483[0] 1772305020.587196 read 0x20ed9c thru 0x211fd1 (0x3236 bytes) 000487[1] 1772305020.613119 mapread 0x342a29 thru 0x343ca7 (0x127f bytes) 000488[1] 1772305020.619936 trunc from 0x8e20cf to 0x60dd70 (0x2d435e bytes) 000489[1] 1772305020.630484 write 0x2ebab9 thru 0x2efcee (0x4236 bytes) 000490[0] 1772305020.639411 read 0x510ae3 thru 0x51a91b (0x9e39 bytes) 000492[0] 1772305020.659101 mapread 0x347ff7 thru 0x355e4c (0xde56 bytes) 000493[1] 1772305020.682139 write 0x8e5bc9 thru 0x8ecd96 (0x71ce bytes) 000494[1] 1772305020.688113 write 0x0ea404 thru 0x0edf64 (0x3b61 bytes) 000495[1] 1772305020.694775 read 0x600bf1 thru 0x608823 (0x7c33 bytes) 000496[0] 1772305020.721185 mapread 0x6f576e thru 0x700117 (0xa9aa bytes) 000497[1] 1772305020.754192 mapwrite 0x533324 thru 0x53ec0c (0xb8e9 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 5020 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x3ab889 000002[0] 1772305021.029202 trunc from 00000000 to 0x3ab888 (0x3ab889 bytes) 000004[1] 1772305021.041895 mapread 0x26a000 thru 0x270233 (0x6234 bytes) truncating to largest ever: 0x658288 000007[0] 1772305021.054105 trunc from 0x3ab889 to 0x658287 (0x2ac9ff bytes) 000009[1] 1772305021.067542 write_OD 0x8d1000 thru 0x8d4fff (0x4000 bytes) 000014[0] 1772305021.085828 read 0x5d0000 thru 0x5de072 (0xe073 bytes) truncating to largest ever: 0x6ef90a 000015[1] 1772305021.088999 trunc from 0x8d5000 to 0x6ef909 (0x1e56f6 bytes) 000016[0] 1772305021.096775 mapwrite 0x3bd000 thru 0x3c98cf (0xc8d0 bytes) 000018[1] 1772305021.144232 read_OD 0x120000 thru 0x123fff (0x4000 bytes) 000020[1] 1772305021.151936 mapwrite 0x674000 thru 0x67a056 (0x6057 bytes) 000022[0] 1772305021.181090 write 0x8ed000 thru 0x8ed4ad (0x4ae bytes) 000026[0] 1772305021.192702 mapwrite 0x67b000 thru 0x683f8b (0x8f8c bytes) 000027[0] 1772305021.218096 write 0x992000 thru 0x998ca3 (0x6ca4 bytes) 000029[1] 1772305021.223101 write_OD 0x89e000 thru 0x8a3fff (0x6000 bytes) 000032[0] 1772305021.242456 mapwrite 0x7d5000 thru 0x7dc390 (0x7391 bytes) 000034[1] 1772305021.273616 mapread 0x935000 thru 0x943d94 (0xed95 bytes) 000035[1] 1772305021.286507 read_OD 0x106000 thru 0x110fff (0xb000 bytes) 000036[0] 1772305021.288935 mapwrite 0x65d000 thru 0x66a4a6 (0xd4a7 bytes) 000037[1] 1772305021.313003 write_OD 0x3d3000 thru 0x3e0fff (0xe000 bytes) 000038[1] 1772305021.327209 write_OD 0x2f7000 thru 0x2fafff (0x4000 bytes) 000042[0] 1772305021.340693 write 0x043000 thru 0x0495b2 (0x65b3 bytes) 000043[1] 1772305021.348059 trunc from 0x998ca4 to 0x178a62 (0x820241 bytes) 000045[1] 1772305021.362796 mapread 0x09c000 thru 0x09fa33 (0x3a34 bytes) 000049[1] 1772305021.377318 mapwrite 0x3d0000 thru 0x3db59b (0xb59c bytes) 000050[0] 1772305021.418614 mapread 0x310000 thru 0x31c3ad (0xc3ae bytes) 000053[0] 1772305021.437279 write 0x164000 thru 0x166219 (0x221a bytes) 000054[0] 1772305021.442541 read 0x02d000 thru 0x0339b9 (0x69ba bytes) 000057[1] 1772305021.488663 mapwrite 0x2a2000 thru 0x2afeab (0xdeac bytes) 000058[0] 1772305021.535709 mapread 0x2e7000 thru 0x2e83ae (0x13af bytes) 000059[0] 1772305021.543424 mapwrite 0x309000 thru 0x31680f (0xd810 bytes) 000062[0] 1772305021.591064 read 0x228000 thru 0x22e742 (0x6743 bytes) truncating to largest ever: 0x775655 000063[0] 1772305021.606713 trunc from 0x3db59c to 0x775654 (0x39a0b9 bytes) 000067[0] 1772305021.618919 mapwrite 0x42f000 thru 0x43b916 (0xc917 bytes) 000068[0] 1772305021.675552 write 0x46f000 thru 0x470a08 (0x1a09 bytes) 000071[1] 1772305021.680201 trunc from 0x775655 to 0x1c3ba8 (0x5b1aac bytes) 000072[0] 1772305021.695444 mapread 0x011000 thru 0x0151bb (0x41bc bytes) 000073[1] 1772305021.706391 trunc from 0x1c3ba9 to 0x2053ee (0x41846 bytes) 000074[1] 1772305021.717166 mapwrite 0x755000 thru 0x75d223 (0x8224 bytes) 000075[1] 1772305021.757839 trunc from 0x75d224 to 0x16fcf2 (0x5ed531 bytes) truncating to largest ever: 0x882487 000078[1] 1772305021.768337 trunc from 0x16fcf3 to 0x882486 (0x712794 bytes) 000079[1] 1772305021.776388 mapread 0x4e1000 thru 0x4eebb5 (0xdbb6 bytes) 000081[0] 1772305021.799226 mapread 0x17d000 thru 0x1862c0 (0x92c1 bytes) 000082[1] 1772305021.819341 write_OD 0x739000 thru 0x740fff (0x8000 bytes) 000085[1] 1772305021.835712 mapwrite 0x4f3000 thru 0x4f92f2 (0x62f3 bytes) 000086[0] 1772305021.860278 mapwrite 0x3a6000 thru 0x3a9ad8 (0x3ad9 bytes) 000087[0] 1772305021.888777 mapread 0x873000 thru 0x880224 (0xd225 bytes) 000088[0] 1772305021.911236 mapread 0x21f000 thru 0x226ec4 (0x7ec5 bytes) 000089[1] 1772305021.925647 read_OD 0x37e000 thru 0x38afff (0xd000 bytes) 000090[0] 1772305021.932771 read 0x003000 thru 0x004d47 (0x1d48 bytes) 000093[0] 1772305022.010856 mapwrite 0x3bc000 thru 0x3c04b9 (0x44ba bytes) 000094[0] 1772305022.049755 trunc from 0x882487 to 0x6fecf9 (0x18378d bytes) 000095[1] 1772305022.056009 read_OD 0x457000 thru 0x464fff (0xe000 bytes) 000097[0] 1772305022.063065 trunc from 0x6fecfa to 0x1b7300 (0x5479f9 bytes) 000098[1] 1772305022.069440 write_OD 0x2a5000 thru 0x2aefff (0xa000 bytes) 000099[1] 1772305022.086879 write_OD 0x8c1000 thru 0x8ccfff (0xc000 bytes) 000100[1] 1772305022.106544 read_OD 0x62f000 thru 0x635fff (0x7000 bytes) 000104[1] 1772305022.111022 mapwrite 0x5c5000 thru 0x5d1263 (0xc264 bytes) 000105[0] 1772305022.167598 mapread 0x570000 thru 0x57d80f (0xd810 bytes) 000106[0] 1772305022.195909 mapread 0x2f8000 thru 0x2fe4d1 (0x64d2 bytes) 000108[0] 1772305022.205916 write 0x65f000 thru 0x664284 (0x5285 bytes) 000109[1] 1772305022.211649 write_OD 0x629000 thru 0x634fff (0xc000 bytes) 000110[1] 1772305022.228054 trunc from 0x8cd000 to 0x2c8171 (0x604e8e bytes) 000115[0] 1772305022.247588 mapread 0x1de000 thru 0x1dee60 (0xe61 bytes) 000118[0] 1772305022.254534 read 0x01b000 thru 0x028d13 (0xdd14 bytes) 000121[0] 1772305022.278968 write 0x700000 thru 0x70e0e3 (0xe0e4 bytes) 000122[1] 1772305022.286571 write_OD 0x364000 thru 0x36efff (0xb000 bytes) 000123[1] 1772305022.304329 mapread 0x664000 thru 0x66a146 (0x6147 bytes) 000124[1] 1772305022.313146 write_OD 0x296000 thru 0x297fff (0x2000 bytes) 000125[1] 1772305022.326604 mapread 0x42c000 thru 0x435ff8 (0x9ff9 bytes) 000126[0] 1772305022.340611 read 0x0ef000 thru 0x0feaec (0xfaed bytes) 000130[1] 1772305022.387994 mapread 0x695000 thru 0x6a10ea (0xc0eb bytes) 000132[0] 1772305022.403816 trunc from 0x70e0e4 to 0x38c518 (0x381bcb bytes) 000133[1] 1772305022.421506 mapread 0x0f4000 thru 0x0f41b5 (0x1b6 bytes) 000135[0] 1772305022.429237 mapwrite 0x26a000 thru 0x275181 (0xb182 bytes) 000138[1] 1772305022.465451 mapwrite 0x541000 thru 0x54b820 (0xa821 bytes) 000139[1] 1772305022.510572 read_OD 0x11b000 thru 0x121fff (0x7000 bytes) 000140[0] 1772305022.516205 trunc from 0x54b821 to 0x2f04d3 (0x25b34d bytes) 000142[1] 1772305022.523723 write_OD 0x547000 thru 0x555fff (0xf000 bytes) 000145[0] 1772305022.546724 trunc from 0x556000 to 0x002dbd (0x553242 bytes) 000148[1] 1772305022.563764 write_OD 0x16f000 thru 0x16ffff (0x1000 bytes) 000152[0] 1772305022.582461 mapread 0x032000 thru 0x038339 (0x633a bytes) 000153[1] 1772305022.592242 mapread 0x106000 thru 0x115a0c (0xfa0d bytes) 000154[1] 1772305022.616359 mapread 0x14d000 thru 0x158cdd (0xbcde bytes) 000155[1] 1772305022.616951 trunc from 0x170000 to 0x0b29af (0xbd650 bytes) 000156[1] 1772305022.625621 mapwrite 0x332000 thru 0x3361ed (0x41ee bytes) 000158[1] 1772305022.651193 mapwrite 0x661000 thru 0x66bc04 (0xac05 bytes) 000160[1] 1772305022.685671 write_OD 0x7ff000 thru 0x803fff (0x5000 bytes) 000162[1] 1772305022.700330 trunc from 0x804000 to 0x29342f (0x570bd0 bytes) 000164[0] 1772305022.710829 mapwrite 0x3c1000 thru 0x3cda6e (0xca6f bytes) 000170[0] 1772305022.762292 trunc from 0x3cda6f to 0x4bc522 (0xeeab4 bytes) 000171[1] 1772305022.771026 write_OD 0x433000 thru 0x438fff (0x6000 bytes) 000172[0] 1772305022.785349 mapread 0x287000 thru 0x28adc6 (0x3dc7 bytes) 000174[0] 1772305022.791515 mapwrite 0x49a000 thru 0x4a3a20 (0x9a21 bytes) 000175[0] 1772305022.824077 write 0x32d000 thru 0x33902e (0xc02f bytes) 000176[0] 1772305022.830048 read 0x234000 thru 0x234deb (0xdec bytes) 000178[1] 1772305022.855934 mapwrite 0x8ff000 thru 0x90c2e9 (0xd2ea bytes) 000179[0] 1772305022.907676 write 0x758000 thru 0x75dd72 (0x5d73 bytes) 000181[0] 1772305022.949390 mapwrite 0x86d000 thru 0x877359 (0xa35a bytes) 000183[0] 1772305022.990636 mapwrite 0x472000 thru 0x473f1d (0x1f1e bytes) 000184[0] 1772305023.013216 mapwrite 0x534000 thru 0x542bf2 (0xebf3 bytes) 000186[0] 1772305023.037051 write 0x69e000 thru 0x6a8c7f (0xac80 bytes) 000187[1] 1772305023.038049 mapwrite 0x377000 thru 0x37f710 (0x8711 bytes) 000190[0] 1772305023.087317 mapwrite 0x1fa000 thru 0x209c64 (0xfc65 bytes) 000191[0] 1772305023.131094 read 0x830000 thru 0x83de6f (0xde70 bytes) 000194[0] 1772305023.144838 read 0x1e1000 thru 0x1e2b94 (0x1b95 bytes) 000195[1] 1772305023.185987 read_OD 0x608000 thru 0x614fff (0xd000 bytes) 000196[0] 1772305023.199970 mapread 0x0dc000 thru 0x0e550f (0x9510 bytes) 000197[1] 1772305023.212346 read_OD 0x011000 thru 0x01afff (0xa000 bytes) 000200[0] 1772305023.214742 mapwrite 0x9eb000 thru 0x9f0f47 (0x5f48 bytes) 000201[0] 1772305023.246036 write 0x697000 thru 0x6a3073 (0xc074 bytes) 000204[0] 1772305023.249112 read 0x639000 thru 0x64748b (0xe48c bytes) 000205[0] 1772305023.275058 read 0x1d7000 thru 0x1df2dd (0x82de bytes) 000208[0] 1772305023.313934 mapread 0x55c000 thru 0x56aa63 (0xea64 bytes) 000209[1] 1772305023.318027 write_OD 0x463000 thru 0x469fff (0x7000 bytes) 000210[1] 1772305023.343174 mapread 0x423000 thru 0x4244f9 (0x14fa bytes) 000213[0] 1772305023.404967 trunc from 0x9f0f48 to 0x125dbe (0x8cb189 bytes) 000214[0] 1772305023.423706 mapwrite 0x31d000 thru 0x328da3 (0xbda4 bytes) 000216[1] 1772305023.467003 mapread 0x01b000 thru 0x024bda (0x9bdb bytes) 000218[1] 1772305023.482723 mapwrite 0x9ff000 thru 0x9ffda0 (0xda1 bytes) 000220[0] 1772305023.515602 mapwrite 0x76b000 thru 0x775d49 (0xad4a bytes) 000221[0] 1772305023.551171 mapread 0x1f3000 thru 0x201a06 (0xea07 bytes) 000222[0] 1772305023.568954 write 0x3d6000 thru 0x3db546 (0x5547 bytes) 000224[1] 1772305023.579542 mapread 0x63b000 thru 0x63b1f6 (0x1f7 bytes) 000225[0] 1772305023.588697 trunc from 0x9ffda1 to 0x5783d4 (0x4879cc bytes) 000226[1] 1772305023.598943 trunc from 0x5783d5 to 0x27ca16 (0x2fb9be bytes) 000227[0] 1772305023.604624 mapwrite 0x216000 thru 0x21dd3c (0x7d3d bytes) 000228[1] 1772305023.638503 mapread 0x269000 thru 0x26fc87 (0x6c88 bytes) 000229[1] 1772305023.651031 mapread 0x0e4000 thru 0x0e9972 (0x5973 bytes) 000230[1] 1772305023.661146 read_OD 0x175000 thru 0x179fff (0x5000 bytes) 000231[0] 1772305023.663400 mapwrite 0x6cc000 thru 0x6d6c37 (0xac38 bytes) 000233[0] 1772305023.711271 write 0x910000 thru 0x918490 (0x8491 bytes) 000234[0] 1772305023.716691 mapwrite 0x764000 thru 0x76aa53 (0x6a54 bytes) 000235[1] 1772305023.741490 trunc from 0x918491 to 0x1439b9 (0x7d4ad7 bytes) truncating to largest ever: 0x9dbba3 000236[0] 1772305023.749619 trunc from 0x1439ba to 0x9dbba2 (0x8981e9 bytes) 000237[0] 1772305023.762653 write 0x38f000 thru 0x399e17 (0xae18 bytes) 000238[0] 1772305023.765588 mapread 0x16b000 thru 0x1767dc (0xb7dd bytes) 000241[0] 1772305023.784164 write 0x43e000 thru 0x4423cc (0x43cd bytes) 000242[0] 1772305023.789673 mapread 0x242000 thru 0x24ab01 (0x8b02 bytes) 000243[0] 1772305023.804270 write 0x73c000 thru 0x74315e (0x715f bytes) 000244[0] 1772305023.806142 read 0x537000 thru 0x543dc3 (0xcdc4 bytes) 000246[1] 1772305023.847452 mapwrite 0x034000 thru 0x0371af (0x31b0 bytes) 000247[1] 1772305023.962826 trunc from 0x9dbba3 to 0x334558 (0x6a764a bytes) 000250[0] 1772305023.979042 mapwrite 0x4d5000 thru 0x4e1427 (0xc428 bytes) 000252[0] 1772305024.030669 trunc from 0x4e1428 to 0x0b5a18 (0x42ba0f bytes) 000255[0] 1772305024.036450 write 0x527000 thru 0x52e049 (0x704a bytes) 000259[0] 1772305024.044832 mapwrite 0x886000 thru 0x89325d (0xd25e bytes) 000262[0] 1772305024.096452 read_OD 0x62f000 thru 0x639fff (0xb000 bytes) 000263[0] 1772305024.101775 write_OD 0x298000 thru 0x2a3fff (0xc000 bytes) 000266[0] 1772305024.118981 mapread 0x25f000 thru 0x266f9b (0x7f9c bytes) 000269[1] 1772305024.124793 mapwrite 0x8a6000 thru 0x8a6686 (0x687 bytes) 000270[1] 1772305024.148828 trunc from 0x8a6687 to 0x6cf5d4 (0x1d70b2 bytes) 000271[0] 1772305024.156882 mapread 0x02c000 thru 0x034dda (0x8ddb bytes) 000272[1] 1772305024.166806 read_OD 0x4d0000 thru 0x4ddfff (0xe000 bytes) 000273[0] 1772305024.170504 mapread 0x438000 thru 0x43f386 (0x7387 bytes) 000274[1] 1772305024.175907 mapwrite 0x243000 thru 0x244186 (0x1187 bytes) 000278[1] 1772305024.223651 write_OD 0x50a000 thru 0x518fff (0xf000 bytes) 000279[1] 1772305024.242590 mapread 0x029000 thru 0x037aff (0xeb00 bytes) 000280[0] 1772305024.261439 trunc from 0x6cf5d5 to 0x9a1d3c (0x2d2768 bytes) 000281[0] 1772305024.273707 trunc from 0x9a1d3d to 0x15d494 (0x8448a8 bytes) 000283[0] 1772305024.278814 trunc from 0x15d495 to 0x0280ab (0x1353e9 bytes) 000284[1] 1772305024.282223 mapwrite 0x4b0000 thru 0x4bf2fb (0xf2fc bytes) 000286[0] 1772305024.328242 trunc from 0x4bf2fc to 0x6bd62f (0x1fe334 bytes) 000287[1] 1772305024.336562 trunc from 0x6bd630 to 0x3b7678 (0x305fb7 bytes) 000290[0] 1772305024.352194 mapwrite 0x297000 thru 0x297ba4 (0xba5 bytes) 000294[1] 1772305024.382574 read_OD 0x21c000 thru 0x225fff (0xa000 bytes) 000296[1] 1772305024.389910 mapwrite 0x54a000 thru 0x559d36 (0xfd37 bytes) 000298[1] 1772305024.447779 mapwrite 0x135000 thru 0x1352f3 (0x2f4 bytes) 000299[1] 1772305024.466492 read_OD 0x137000 thru 0x13afff (0x4000 bytes) 000303[0] 1772305024.478694 mapread 0x02d000 thru 0x034225 (0x7226 bytes) 000305[1] 1772305024.484254 read_OD 0x29d000 thru 0x2abfff (0xf000 bytes) 000306[0] 1772305024.486761 mapread 0x4df000 thru 0x4edb47 (0xeb48 bytes) 000312[1] 1772305024.496565 trunc from 0x559d37 to 0x415c1e (0x144118 bytes) 000315[0] 1772305024.510951 trunc from 0x415c1f to 0x87a174 (0x464556 bytes) 000316[1] 1772305024.519610 trunc from 0x87a175 to 0x04008a (0x83a0ea bytes) 000318[1] 1772305024.531485 mapread 0x031000 thru 0x03a710 (0x9711 bytes) 000319[0] 1772305024.543709 mapwrite 0x342000 thru 0x34c451 (0xa452 bytes) 000320[0] 1772305024.574366 mapread 0x0e5000 thru 0x0ef55f (0xa560 bytes) 000322[1] 1772305024.584353 mapread 0x0b0000 thru 0x0b41ec (0x41ed bytes) 000323[1] 1772305024.602166 mapwrite 0x291000 thru 0x2997a6 (0x87a7 bytes) 000325[1] 1772305024.635118 mapwrite 0x207000 thru 0x2089c7 (0x19c8 bytes) 000326[1] 1772305024.654719 read_OD 0x284000 thru 0x28efff (0xb000 bytes) 000328[1] 1772305024.659811 read_OD 0x185000 thru 0x18dfff (0x9000 bytes) 000331[1] 1772305024.663836 trunc from 0x34c452 to 0x80cd25 (0x4c08d4 bytes) 000333[1] 1772305024.678585 write_OD 0x16e000 thru 0x177fff (0xa000 bytes) 000334[0] 1772305024.693072 read_OD 0x510000 thru 0x51bfff (0xc000 bytes) 000338[1] 1772305024.697641 mapread 0x268000 thru 0x26e77b (0x677c bytes) 000339[0] 1772305024.741929 write_OD 0x6a0000 thru 0x6a0fff (0x1000 bytes) 000348[1] 1772305024.766202 trunc from 0x80cd26 to 0x47f6b0 (0x38d675 bytes) 000350[1] 1772305024.781513 read_OD 0x020000 thru 0x025fff (0x6000 bytes) 000351[0] 1772305024.787761 mapwrite 0x0b3000 thru 0x0bc9de (0x99df bytes) 000352[0] 1772305024.822975 mapread 0x395000 thru 0x3a3346 (0xe347 bytes) 000353[0] 1772305024.840548 mapwrite 0x560000 thru 0x56971e (0x971f bytes) 000354[0] 1772305024.864485 trunc from 0x56971f to 0x44bc0d (0x11db11 bytes) 000356[0] 1772305024.875967 write_OD 0x042000 thru 0x050fff (0xf000 bytes) 000357[1] 1772305024.889292 mapread 0x23d000 thru 0x23fafe (0x2aff bytes) 000359[0] 1772305024.921601 trunc from 0x44bc0e to 0x63a3e3 (0x1ee7d6 bytes) 000361[1] 1772305024.938154 read_OD 0x404000 thru 0x406fff (0x3000 bytes) 000363[0] 1772305024.944766 read_OD 0x4ea000 thru 0x4f6fff (0xd000 bytes) 000364[1] 1772305024.947056 write_OD 0x213000 thru 0x21ffff (0xd000 bytes) 000365[1] 1772305024.960540 trunc from 0x63a3e4 to 0x6c1ed8 (0x87af5 bytes) 000367[0] 1772305024.975517 mapwrite 0x5e2000 thru 0x5e27f4 (0x7f5 bytes) 000368[0] 1772305025.007302 mapread 0x6a5000 thru 0x6ae806 (0x9807 bytes) 000373[1] 1772305025.020283 read_OD 0x4fc000 thru 0x506fff (0xb000 bytes) 000376[0] 1772305025.030198 write_OD 0x8e0000 thru 0x8e7fff (0x8000 bytes) 000377[1] 1772305025.046073 mapread 0x611000 thru 0x61eeab (0xdeac bytes) 000382[0] 1772305025.080770 mapread 0x775000 thru 0x776636 (0x1637 bytes) 000387[1] 1772305025.084567 trunc from 0x8e8000 to 0x0d9470 (0x80eb8f bytes) 000391[1] 1772305025.102396 read_OD 0x03e000 thru 0x044fff (0x7000 bytes) 000394[1] 1772305025.108436 write_OD 0x3a6000 thru 0x3b3fff (0xe000 bytes) 000395[1] 1772305025.126488 read_OD 0x1ad000 thru 0x1b3fff (0x7000 bytes) 000396[1] 1772305025.130287 mapwrite 0x247000 thru 0x24b681 (0x4682 bytes) 000397[0] 1772305025.166838 write_OD 0x7f9000 thru 0x7f9fff (0x1000 bytes) 000398[0] 1772305025.182650 mapwrite 0x31b000 thru 0x3224db (0x74dc bytes) 000399[1] 1772305025.209976 write_OD 0x632000 thru 0x632fff (0x1000 bytes) 000401[1] 1772305025.222609 mapread 0x6f6000 thru 0x6fd24d (0x724e bytes) 000402[1] 1772305025.232732 trunc from 0x7fa000 to 0x30dedb (0x4ec124 bytes) 000403[0] 1772305025.241221 trunc from 0x30dedc to 0x718c8f (0x40adb4 bytes) 000404[0] 1772305025.252752 trunc from 0x718c90 to 0x5265b8 (0x1f26d7 bytes) 000406[1] 1772305025.264912 read_OD 0x338000 thru 0x33efff (0x7000 bytes) 000407[0] 1772305025.272629 read_OD 0x15f000 thru 0x16cfff (0xe000 bytes) 000409[0] 1772305025.276290 read_OD 0x433000 thru 0x43cfff (0xa000 bytes) 000410[0] 1772305025.278547 trunc from 0x5265b9 to 0x420916 (0x105ca2 bytes) 000411[1] 1772305025.285981 mapwrite 0x6e7000 thru 0x6f6862 (0xf863 bytes) 000412[1] 1772305025.333144 mapread 0x1c2000 thru 0x1c4b61 (0x2b62 bytes) 000415[1] 1772305025.342822 mapread 0x1b5000 thru 0x1b9617 (0x4618 bytes) 000416[0] 1772305025.351982 mapread 0x6f4000 thru 0x6f5d7e (0x1d7f bytes) 000417[1] 1772305025.384149 read_OD 0x305000 thru 0x309fff (0x5000 bytes) 000419[1] 1772305025.386722 trunc from 0x6f6863 to 0x252641 (0x4a4221 bytes) 000426[0] 1772305025.420940 write_OD 0x3b8000 thru 0x3b9fff (0x2000 bytes) 000427[0] 1772305025.435813 mapread 0x1f8000 thru 0x1fb23b (0x323c bytes) 000430[1] 1772305025.443825 mapread 0x25b000 thru 0x267b0b (0xcb0c bytes) 000431[1] 1772305025.451576 trunc from 0x3ba000 to 0x66fb4d (0x2b5b4e bytes) 000435[0] 1772305025.466008 read_OD 0x4ba000 thru 0x4c1fff (0x8000 bytes) 000436[1] 1772305025.472978 mapwrite 0x486000 thru 0x487b9a (0x1b9b bytes) 000437[1] 1772305025.491439 mapread 0x2d8000 thru 0x2de9b5 (0x69b6 bytes) 000438[0] 1772305025.501146 trunc from 0x66fb4e to 0x8561da (0x1e668d bytes) 000442[0] 1772305025.518058 trunc from 0x8561db to 0x6477d4 (0x20ea06 bytes) 000445[1] 1772305025.523202 write_OD 0x187000 thru 0x194fff (0xe000 bytes) 000446[0] 1772305025.540904 write_OD 0x8ac000 thru 0x8affff (0x4000 bytes) 000449[1] 1772305025.558342 mapread 0x4cd000 thru 0x4d224a (0x524b bytes) 000450[0] 1772305025.563346 mapwrite 0x322000 thru 0x322154 (0x155 bytes) 000451[1] 1772305025.625188 mapread 0x064000 thru 0x0739dc (0xf9dd bytes) 000452[0] 1772305025.641309 trunc from 0x8b0000 to 0x436cc5 (0x47933a bytes) 000454[0] 1772305025.649415 mapread 0x0cd000 thru 0x0d03c6 (0x33c7 bytes) 000455[1] 1772305025.659066 read_OD 0x315000 thru 0x318fff (0x4000 bytes) 000456[1] 1772305025.664895 trunc from 0x436cc6 to 0x461d57 (0x2b092 bytes) 000457[0] 1772305025.671408 write_OD 0x9f4000 thru 0x9fcfff (0x9000 bytes) 000458[1] 1772305025.690523 mapwrite 0x473000 thru 0x47cf46 (0x9f47 bytes) 000461[1] 1772305025.726903 mapwrite 0x812000 thru 0x8185ee (0x65ef bytes) 000462[1] 1772305025.748072 trunc from 0x9fd000 to 0x929cf3 (0xd330c bytes) 000463[1] 1772305025.753610 mapread 0x913000 thru 0x91d663 (0xa664 bytes) 000466[1] 1772305025.764105 write_OD 0x5cc000 thru 0x5d8fff (0xd000 bytes) 000468[0] 1772305025.779985 mapread 0x6ef000 thru 0x6fc1ee (0xd1ef bytes) 000472[1] 1772305025.802427 write_OD 0x0fb000 thru 0x103fff (0x9000 bytes) 000473[0] 1772305025.819231 trunc from 0x929cf4 to 0x5d6d2f (0x352fc4 bytes) 000474[0] 1772305025.831232 mapwrite 0x42a000 thru 0x43518e (0xb18f bytes) 000476[1] 1772305025.871179 mapread 0x4a2000 thru 0x4a3807 (0x1808 bytes) 000477[0] 1772305025.880411 mapread 0x1be000 thru 0x1c635a (0x835b bytes) 000479[1] 1772305025.891920 mapwrite 0x689000 thru 0x68a58d (0x158e bytes) 000481[0] 1772305025.926216 mapwrite 0x615000 thru 0x6182c8 (0x32c9 bytes) 000483[1] 1772305025.958536 trunc from 0x68a58e to 0x2632b8 (0x4272d5 bytes) 000486[0] 1772305025.972491 mapread 0x0a9000 thru 0x0b36a5 (0xa6a6 bytes) 000488[0] 1772305025.992124 trunc from 0x2632b9 to 0x717fb8 (0x4b4d00 bytes) 000490[0] 1772305026.009976 mapread 0x65a000 thru 0x66030a (0x630b bytes) 000491[0] 1772305026.019947 mapread 0x452000 thru 0x459c58 (0x7c59 bytes) 000492[0] 1772305026.029674 mapread 0x08a000 thru 0x08ce8a (0x2e8b bytes) 000497[0] 1772305026.047943 mapwrite 0x275000 thru 0x277d99 (0x2d9a bytes) 000498[0] 1772305026.069081 mapread 0x58d000 thru 0x597b12 (0xab13 bytes) 000499[0] 1772305026.086939 mapwrite 0x302000 thru 0x30f185 (0xd186 bytes) 000500[0] 1772305026.123023 mapwrite 0x503000 thru 0x50f084 (0xc085 bytes) All operations completed A-OK! PASS 16a (14s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 13:57:07 (1772305027) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0844562 s, 24.8 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0655497 s, 32.0 MB/s Chance of close/open is 1 in 50 Seed set to 5028 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[1] 1772305028.817590 write 0x2c107c thru 0x2c2696 (0x161b bytes) 000002[0] 1772305028.827499 read 0x2370bd thru 0x2378c3 (0x807 bytes) 000003[0] 1772305028.837156 mapwrite 0x90ee57 thru 0x910bc1 (0x1d6b bytes) 000005[0] 1772305028.899938 write 0x514dda thru 0x515652 (0x879 bytes) truncating to largest ever: 0x7a3722 000006[1] 1772305028.903178 trunc from 0x910bc2 to 0x7a3721 (0x16d4a0 bytes) 000007[1] 1772305028.915131 mapread 0x1cd165 thru 0x1cef78 (0x1e14 bytes) 000008[0] 1772305028.918664 mapwrite 0x5efa24 thru 0x5f0322 (0x8ff bytes) 000009[0] 1772305028.944085 mapread 0x086a16 thru 0x087f0d (0x14f8 bytes) 000010[1] 1772305028.952326 write 0x418c7f thru 0x41a9f3 (0x1d75 bytes) 000011[1] 1772305028.960707 trunc from 0x7a3722 to 0x73c3ff (0x67322 bytes) 000012[1] 1772305028.972987 mapwrite 0x3290eb thru 0x32afae (0x1ec4 bytes) 000015[1] 1772305028.996185 mapread 0x3d6f65 thru 0x3d8935 (0x19d1 bytes) 000018[0] 1772305029.008230 mapread 0x53e762 thru 0x53fa00 (0x129f bytes) 000020[1] 1772305029.016796 mapwrite 0x566b7d thru 0x5674ae (0x932 bytes) 000021[1] 1772305029.040153 write 0x9145e0 thru 0x915c10 (0x1631 bytes) 000022[1] 1772305029.041445 write 0x58c351 thru 0x58e2e0 (0x1f90 bytes) truncating to largest ever: 0x960383 000023[1] 1772305029.051280 trunc from 0x915c11 to 0x960382 (0x4a772 bytes) 000025[1] 1772305029.062142 trunc from 0x960383 to 0x0bff9f (0x8a03e3 bytes) 000026[0] 1772305029.068588 write 0x47ed73 thru 0x47fd96 (0x1024 bytes) 000027[0] 1772305029.076381 read 0x3f3519 thru 0x3f4711 (0x11f9 bytes) 000028[1] 1772305029.085940 mapread 0x17f079 thru 0x18026f (0x11f7 bytes) 000029[1] 1772305029.095618 mapwrite 0x316b86 thru 0x318199 (0x1614 bytes) 000030[1] 1772305029.134594 mapwrite 0x592497 thru 0x5935a6 (0x1110 bytes) 000031[1] 1772305029.160321 mapwrite 0x689234 thru 0x689fc2 (0xd8f bytes) 000032[1] 1772305029.203303 mapwrite 0x944017 thru 0x9454a9 (0x1493 bytes) 000035[0] 1772305029.226739 trunc from 0x9454aa to 0x8afd67 (0x95742 bytes) 000036[0] 1772305029.236992 mapwrite 0x6352e3 thru 0x6359dd (0x6fb bytes) 000038[1] 1772305029.286227 mapwrite 0x115a31 thru 0x116226 (0x7f6 bytes) 000039[0] 1772305029.315595 mapread 0x860bcd thru 0x862609 (0x1a3d bytes) 000042[1] 1772305029.331156 read 0x2adfce thru 0x2af6f0 (0x1723 bytes) 000044[1] 1772305029.346083 read 0x6865b3 thru 0x686801 (0x24f bytes) 000047[0] 1772305029.375028 read 0x27a764 thru 0x27c6be (0x1f5b bytes) 000048[1] 1772305029.385145 read 0x7d7841 thru 0x7d93a3 (0x1b63 bytes) 000050[1] 1772305029.389253 mapread 0x2ce054 thru 0x2ceb75 (0xb22 bytes) 000051[0] 1772305029.391188 trunc from 0x8afd68 to 0x7e53c3 (0xca9a4 bytes) 000052[0] 1772305029.411281 trunc from 0x7e53c4 to 0x30ecb6 (0x4d670d bytes) 000054[1] 1772305029.416656 mapwrite 0x3f9495 thru 0x3f95c5 (0x131 bytes) 000055[0] 1772305029.442886 mapwrite 0x7cd4ef thru 0x7ce7f8 (0x130a bytes) 000056[1] 1772305029.517095 mapwrite 0x94c2c6 thru 0x94da22 (0x175d bytes) 000058[0] 1772305029.551808 mapwrite 0x7140b9 thru 0x715448 (0x1390 bytes) 000060[0] 1772305029.579635 trunc from 0x94da23 to 0x914cb7 (0x38d6b bytes) 000061[0] 1772305029.591120 mapread 0x0d9f56 thru 0x0db8e4 (0x198f bytes) 000062[1] 1772305029.598974 mapread 0x34dceb thru 0x34f639 (0x194f bytes) 000064[1] 1772305029.613099 trunc from 0x914cb8 to 0x8c1899 (0x5341e bytes) 000065[0] 1772305029.622162 mapread 0x0fb91e thru 0x0fc485 (0xb68 bytes) 000069[0] 1772305029.638459 mapwrite 0x0e5bf4 thru 0x0e5ea8 (0x2b5 bytes) 000071[1] 1772305029.665110 write 0x07b5d4 thru 0x07b931 (0x35e bytes) 000072[1] 1772305029.669303 mapwrite 0x5b526e thru 0x5b5cc0 (0xa53 bytes) 000073[0] 1772305029.684000 write 0x220c93 thru 0x2229da (0x1d48 bytes) 000074[1] 1772305029.693970 read 0x790f5d thru 0x790fa5 (0x49 bytes) 000077[0] 1772305029.711170 write 0x474653 thru 0x47544b (0xdf9 bytes) 000078[0] 1772305029.717839 trunc from 0x8c189a to 0x611af7 (0x2afda2 bytes) 000079[1] 1772305029.726212 write 0x650c35 thru 0x652174 (0x1540 bytes) 000082[0] 1772305029.742998 mapwrite 0x85980b thru 0x85b069 (0x185f bytes) 000084[0] 1772305029.777071 mapwrite 0x59f8d5 thru 0x5a0198 (0x8c4 bytes) 000085[0] 1772305029.792751 write 0x524bcf thru 0x526613 (0x1a45 bytes) 000087[1] 1772305029.804889 mapread 0x013079 thru 0x01343b (0x3c3 bytes) 000092[1] 1772305029.826296 read 0x3c4f20 thru 0x3c53ca (0x4ab bytes) 000093[1] 1772305029.860762 trunc from 0x85b06a to 0x01ebff (0x83c46a bytes) 000095[0] 1772305029.886306 write 0x8f98c3 thru 0x8fb7b4 (0x1ef2 bytes) 000096[0] 1772305029.895794 write 0x62d189 thru 0x62dade (0x956 bytes) 000097[1] 1772305029.902598 write 0x100d0b thru 0x102aee (0x1de4 bytes) 000098[0] 1772305029.906691 mapread 0x0d4fad thru 0x0d5b89 (0xbdd bytes) 000099[1] 1772305029.914940 trunc from 0x8fb7b5 to 0x885a18 (0x75d9c bytes) 000100[0] 1772305029.929298 read 0x736665 thru 0x73672f (0xcb bytes) truncating to largest ever: 0x9db8e2 000101[1] 1772305029.944085 trunc from 0x885a19 to 0x9db8e1 (0x155ec9 bytes) 000102[1] 1772305029.956586 read 0x246003 thru 0x2475b0 (0x15ae bytes) 000103[0] 1772305030.029845 write 0x23fa29 thru 0x24169b (0x1c73 bytes) 000105[0] 1772305030.054959 trunc from 0x9db8e2 to 0x812a45 (0x1c8e9c bytes) 000106[0] 1772305030.066061 trunc from 0x812a46 to 0x87192c (0x5eee7 bytes) 000107[1] 1772305030.070165 write 0x9d6c8f thru 0x9d7b1b (0xe8d bytes) 000109[1] 1772305030.082032 read 0x8cb72a thru 0x8cb781 (0x58 bytes) 000110[1] 1772305030.096899 mapread 0x54117d thru 0x5422da (0x115e bytes) 000111[1] 1772305030.103338 trunc from 0x9d7b1c to 0x30901e (0x6ceafd bytes) 000112[1] 1772305030.116350 trunc from 0x30901f to 0x18e7c6 (0x17a858 bytes) 000113[1] 1772305030.121366 mapread 0x10c4ba thru 0x10dcf1 (0x1838 bytes) 000116[0] 1772305030.129971 read 0x0f5c15 thru 0x0f5f32 (0x31e bytes) 000117[1] 1772305030.138256 mapwrite 0x414c92 thru 0x416001 (0x1370 bytes) 000118[1] 1772305030.166766 trunc from 0x416002 to 0x6954bc (0x27f4bb bytes) 000122[0] 1772305030.173325 read 0x3a936e thru 0x3aafb9 (0x1c4c bytes) 000124[0] 1772305030.181307 mapwrite 0x9275f1 thru 0x92845b (0xe6b bytes) 000127[1] 1772305030.241208 mapread 0x46656e thru 0x467619 (0x10ac bytes) 000129[1] 1772305030.252993 write 0x3c72fb thru 0x3c9024 (0x1d2a bytes) 000132[1] 1772305030.264906 write 0x15da59 thru 0x15e54c (0xaf4 bytes) 000136[1] 1772305030.276281 write 0x5eb9ea thru 0x5ec908 (0xf1f bytes) 000139[0] 1772305030.291704 read 0x1535b7 thru 0x153689 (0xd3 bytes) 000140[1] 1772305030.345972 mapread 0x68e477 thru 0x68f375 (0xeff bytes) 000141[1] 1772305030.349141 trunc from 0x92845c to 0x3f8d61 (0x52f6fa bytes) 000142[0] 1772305030.366569 mapread 0x162ab5 thru 0x163135 (0x681 bytes) 000143[0] 1772305030.378029 mapwrite 0x0cda12 thru 0x0ce7f6 (0xde5 bytes) 000147[1] 1772305030.400298 write 0x96f320 thru 0x970b05 (0x17e6 bytes) 000149[1] 1772305030.412102 trunc from 0x970b06 to 0x05860a (0x9184fb bytes) 000155[0] 1772305030.441219 mapread 0x0402a2 thru 0x041eda (0x1c39 bytes) 000156[0] 1772305030.449141 read 0x0465db thru 0x04734b (0xd71 bytes) 000157[1] 1772305030.454941 mapwrite 0x215c6e thru 0x21608a (0x41d bytes) 000159[0] 1772305030.476531 trunc from 0x21608b to 0x424716 (0x20e68c bytes) 000160[0] 1772305030.490184 mapread 0x1d198e thru 0x1d3739 (0x1dac bytes) 000163[0] 1772305030.525780 mapread 0x23978c thru 0x23b162 (0x19d7 bytes) 000165[0] 1772305030.534160 read 0x3cc43e thru 0x3cd9cc (0x158f bytes) 000166[0] 1772305030.534661 mapread 0x121264 thru 0x1222e6 (0x1083 bytes) truncating to largest ever: 0x9f72a2 000168[1] 1772305030.543299 trunc from 0x424717 to 0x9f72a1 (0x5d2b8b bytes) 000171[0] 1772305030.564442 trunc from 0x9f72a2 to 0x104280 (0x8f3021 bytes) 000172[1] 1772305030.578373 write 0x0da1f8 thru 0x0da258 (0x61 bytes) 000173[1] 1772305030.586344 mapread 0x10268a thru 0x1026eb (0x62 bytes) 000174[0] 1772305030.591646 mapwrite 0x0b5174 thru 0x0b5cf3 (0xb80 bytes) 000175[1] 1772305030.616533 mapread 0x0ee54d thru 0x0eef86 (0xa3a bytes) 000176[0] 1772305030.622490 read 0x02714e thru 0x027255 (0x108 bytes) 000177[0] 1772305030.632717 write 0x19d383 thru 0x19dc02 (0x880 bytes) 000178[0] 1772305030.639437 trunc from 0x19dc03 to 0x0b6071 (0xe7b91 bytes) 000179[1] 1772305030.647718 write 0x71eb1c thru 0x71fc96 (0x117b bytes) 000181[1] 1772305030.660671 read 0x33d5dc thru 0x33e891 (0x12b6 bytes) 000182[1] 1772305030.669816 trunc from 0x71fc97 to 0x5c32be (0x15c9d8 bytes) 000185[0] 1772305030.686282 mapwrite 0x6d9eb4 thru 0x6daddc (0xf29 bytes) 000186[1] 1772305030.749359 mapread 0x549c6b thru 0x54b7d9 (0x1b6f bytes) 000188[0] 1772305030.766559 mapwrite 0x76f8b2 thru 0x7705e0 (0xd2f bytes) 000189[0] 1772305030.797185 mapread 0x04ce6e thru 0x04e9d8 (0x1b6b bytes) 000190[1] 1772305030.802547 write 0x8ce282 thru 0x8cea24 (0x7a3 bytes) 000191[1] 1772305030.812395 mapwrite 0x978ab7 thru 0x9796f6 (0xc40 bytes) 000194[0] 1772305030.871805 mapwrite 0x053cca thru 0x055bae (0x1ee5 bytes) 000196[0] 1772305030.906050 mapwrite 0x199496 thru 0x19b2dc (0x1e47 bytes) 000198[1] 1772305030.936520 trunc from 0x9796f7 to 0x30c038 (0x66d6be bytes) 000199[1] 1772305030.946460 mapwrite 0x532293 thru 0x5335cc (0x133a bytes) 000200[1] 1772305030.970517 mapread 0x48f61e thru 0x49030c (0xcef bytes) 000203[0] 1772305030.979225 trunc from 0x5335cd to 0x8c3b8d (0x3905c1 bytes) 000204[1] 1772305030.994792 read 0x74c98a thru 0x74dded (0x1464 bytes) 000205[1] 1772305031.007210 read 0x4a076c thru 0x4a0e6e (0x703 bytes) 000209[0] 1772305031.036382 read 0x69262f thru 0x693781 (0x1153 bytes) 000213[0] 1772305031.051615 mapread 0x206ca1 thru 0x206fe7 (0x347 bytes) 000214[1] 1772305031.053595 mapwrite 0x74bcb6 thru 0x74ca9e (0xde9 bytes) 000215[0] 1772305031.091665 write 0x47042f thru 0x471078 (0xc4a bytes) 000216[1] 1772305031.109658 read 0x350cb6 thru 0x35222f (0x157a bytes) 000217[1] 1772305031.119425 read 0x73ed5f thru 0x73f2c2 (0x564 bytes) 000219[0] 1772305031.133815 trunc from 0x8c3b8e to 0x1da668 (0x6e9525 bytes) 000224[1] 1772305031.152571 write 0x584dfa thru 0x58524d (0x454 bytes) 000227[1] 1772305031.167548 read 0x4c11e8 thru 0x4c12d5 (0xee bytes) 000228[0] 1772305031.175290 mapwrite 0x0760aa thru 0x0773d0 (0x1327 bytes) 000229[0] 1772305031.198603 write 0x3d1f43 thru 0x3d3a37 (0x1af5 bytes) 000232[1] 1772305031.206412 trunc from 0x58524e to 0x851453 (0x2cc206 bytes) 000233[1] 1772305031.219017 mapwrite 0x133988 thru 0x134906 (0xf7f bytes) 000236[1] 1772305031.267833 write 0x179ba9 thru 0x17b93a (0x1d92 bytes) 000237[1] 1772305031.275362 trunc from 0x851454 to 0x4e2a15 (0x36ea3e bytes) 000238[0] 1772305031.288362 read 0x3e26a4 thru 0x3e2d8d (0x6ea bytes) 000239[0] 1772305031.299548 mapwrite 0x96a06e thru 0x96a0f8 (0x8b bytes) 000244[1] 1772305031.389550 trunc from 0x96a0f9 to 0x272c76 (0x6f7482 bytes) 000245[0] 1772305031.410494 write 0x113f60 thru 0x115ad7 (0x1b78 bytes) 000246[1] 1772305031.418674 mapwrite 0x3d30a7 thru 0x3d312c (0x86 bytes) 000247[0] 1772305031.451782 mapwrite 0x76b638 thru 0x76c867 (0x1230 bytes) 000250[1] 1772305031.488204 write 0x183c4e thru 0x1857ba (0x1b6d bytes) 000252[1] 1772305031.500574 read 0x2fd5a6 thru 0x2feb63 (0x15be bytes) 000254[1] 1772305031.554674 mapwrite 0x00e5ef thru 0x00fc7d (0x168f bytes) 000258[0] 1772305031.588528 trunc from 0x76c868 to 0x6cbc17 (0xa0c50 bytes) 000260[1] 1772305031.608503 mapread 0x5948aa thru 0x595ae9 (0x1240 bytes) 000261[0] 1772305031.621142 mapread 0x4cfd7f thru 0x4d0d37 (0xfb9 bytes) 000264[0] 1772305031.630975 mapwrite 0x2c4bba thru 0x2c6571 (0x19b8 bytes) 000265[0] 1772305031.651654 read 0x3e8971 thru 0x3e9f5c (0x15ec bytes) 000269[1] 1772305031.686187 mapwrite 0x9b1210 thru 0x9b2eec (0x1cdd bytes) 000270[1] 1772305031.740269 mapread 0x7787f3 thru 0x77a507 (0x1d15 bytes) 000271[1] 1772305031.745176 mapread 0x4b807c thru 0x4b90d6 (0x105b bytes) 000272[1] 1772305031.752554 mapwrite 0x9b965b thru 0x9b9c57 (0x5fd bytes) 000273[0] 1772305031.772310 mapwrite 0x3958ee thru 0x396cd8 (0x13eb bytes) 000278[1] 1772305031.813827 mapwrite 0x9ff04a thru 0x9fffff (0xfb6 bytes) 000279[0] 1772305031.835787 mapwrite 0x189150 thru 0x18a5cd (0x147e bytes) 000281[1] 1772305031.863172 write 0x0fabae thru 0x0fc012 (0x1465 bytes) 000284[0] 1772305031.878899 mapread 0x260ca5 thru 0x262b27 (0x1e83 bytes) 000287[1] 1772305031.931947 write 0x3fd0a6 thru 0x3fea9e (0x19f9 bytes) 000288[0] 1772305031.943185 mapwrite 0x9c332a thru 0x9c394e (0x625 bytes) 000289[0] 1772305031.963023 mapwrite 0x18bba0 thru 0x18d659 (0x1aba bytes) 000291[0] 1772305031.988306 trunc from 0xa00000 to 0x2dca8c (0x723573 bytes) 000294[1] 1772305032.007222 mapread 0x249898 thru 0x24a05d (0x7c6 bytes) 000296[1] 1772305032.019604 write 0x549df1 thru 0x549f5e (0x16e bytes) 000297[1] 1772305032.023790 mapread 0x4daea0 thru 0x4dbad9 (0xc3a bytes) 000298[1] 1772305032.031073 read 0x3679c2 thru 0x368fe4 (0x1623 bytes) 000299[1] 1772305032.052926 mapread 0x3b40e7 thru 0x3b52cf (0x11e9 bytes) 000300[1] 1772305032.057087 mapread 0x1226cc thru 0x122737 (0x6c bytes) 000301[0] 1772305032.058796 mapwrite 0x72eb64 thru 0x72f283 (0x720 bytes) 000302[1] 1772305032.092230 read 0x0142db thru 0x015dc2 (0x1ae8 bytes) 000303[0] 1772305032.108016 write 0x93679f thru 0x936e89 (0x6eb bytes) 000304[0] 1772305032.111839 mapread 0x5baab8 thru 0x5bc7ef (0x1d38 bytes) 000305[1] 1772305032.116041 write 0x2b7272 thru 0x2b782c (0x5bb bytes) 000306[1] 1772305032.123709 mapread 0x604625 thru 0x604e5e (0x83a bytes) 000307[0] 1772305032.129885 mapread 0x651c28 thru 0x6521c5 (0x59e bytes) 000308[0] 1772305032.137340 write 0x19fff9 thru 0x1a120d (0x1215 bytes) 000309[1] 1772305032.143196 read 0x8ed213 thru 0x8ef173 (0x1f61 bytes) 000310[0] 1772305032.153488 read 0x022136 thru 0x022321 (0x1ec bytes) 000311[0] 1772305032.221456 read 0x571ccc thru 0x5738d1 (0x1c06 bytes) 000317[1] 1772305032.252720 write 0x662dfd thru 0x6634bb (0x6bf bytes) 000318[0] 1772305032.268123 mapwrite 0x825130 thru 0x825fba (0xe8b bytes) 000320[1] 1772305032.287701 mapwrite 0x67a02d thru 0x67af01 (0xed5 bytes) 000321[0] 1772305032.310014 trunc from 0x936e8a to 0x7d02ba (0x166bcf bytes) 000323[0] 1772305032.326473 trunc from 0x7d02bb to 0x205bf2 (0x5ca6c8 bytes) 000324[1] 1772305032.334322 mapread 0x1ded3e thru 0x1e08ec (0x1baf bytes) 000327[1] 1772305032.354628 mapread 0x121d66 thru 0x122aa2 (0xd3d bytes) 000328[1] 1772305032.359196 mapread 0x024293 thru 0x025cc4 (0x1a32 bytes) 000329[1] 1772305032.366585 trunc from 0x205bf3 to 0x33d4fc (0x13790a bytes) 000330[1] 1772305032.381000 mapread 0x0d9646 thru 0x0d9b42 (0x4fd bytes) 000331[1] 1772305032.383740 mapwrite 0x0e7c66 thru 0x0e87a7 (0xb42 bytes) 000332[1] 1772305032.403232 mapwrite 0x7b6e5f thru 0x7b8d2c (0x1ece bytes) 000333[0] 1772305032.431479 trunc from 0x7b8d2d to 0x0efad5 (0x6c9257 bytes) 000334[1] 1772305032.443846 read 0x08ca7a thru 0x08d0b2 (0x639 bytes) 000338[0] 1772305032.466108 mapwrite 0x5b80ff thru 0x5b99b9 (0x18bb bytes) 000344[1] 1772305032.505271 mapread 0x54d5e4 thru 0x54f3e8 (0x1e05 bytes) 000345[1] 1772305032.514318 read 0x202dde thru 0x20394c (0xb6f bytes) 000348[1] 1772305032.550762 trunc from 0x5b99ba to 0x003a0c (0x5b5fad bytes) 000349[1] 1772305032.564891 read 0x003831 thru 0x003a0c (0x1dc bytes) 000352[1] 1772305032.592242 mapread 0x000004 thru 0x000f33 (0xf30 bytes) 000355[0] 1772305032.595666 mapwrite 0x090729 thru 0x0908e2 (0x1ba bytes) 000356[1] 1772305032.627651 mapread 0x08a97d thru 0x08ae0e (0x492 bytes) 000358[0] 1772305032.640936 trunc from 0x0908e3 to 0x4abb8b (0x41b2a9 bytes) 000359[1] 1772305032.649545 mapwrite 0x317dd1 thru 0x3199ec (0x1c1c bytes) 000361[0] 1772305032.681545 read 0x1abf1d thru 0x1ad507 (0x15eb bytes) 000362[0] 1772305032.713056 read 0x344f44 thru 0x3455a7 (0x664 bytes) 000363[1] 1772305032.717556 trunc from 0x4abb8c to 0x17aecd (0x330cbe bytes) 000364[0] 1772305032.732330 write 0x8dea26 thru 0x8e0066 (0x1641 bytes) 000366[1] 1772305032.746988 mapwrite 0x1dc0f3 thru 0x1dda40 (0x194e bytes) 000367[1] 1772305032.775719 read 0x687798 thru 0x6886c7 (0xf30 bytes) 000368[0] 1772305032.782781 mapread 0x304778 thru 0x305d3e (0x15c7 bytes) 000372[1] 1772305032.791991 mapwrite 0x74e82e thru 0x74f0d5 (0x8a8 bytes) 000373[1] 1772305032.809373 trunc from 0x8e0067 to 0x344051 (0x59c015 bytes) 000375[0] 1772305032.822383 write 0x1290ad thru 0x129ddb (0xd2f bytes) 000376[0] 1772305032.829921 trunc from 0x344052 to 0x2ccd2b (0x77326 bytes) 000379[0] 1772305032.857245 mapread 0x1e174f thru 0x1e29f2 (0x12a4 bytes) 000381[0] 1772305032.869376 trunc from 0x2ccd2c to 0x7ae18a (0x4e145f bytes) 000382[1] 1772305032.876418 write 0x6d8c1d thru 0x6da872 (0x1c56 bytes) 000383[0] 1772305032.885006 mapread 0x6bf20f thru 0x6c0298 (0x108a bytes) 000384[0] 1772305032.898585 write 0x47c942 thru 0x47df62 (0x1621 bytes) 000385[0] 1772305032.907440 mapwrite 0x285cb6 thru 0x28650d (0x858 bytes) 000386[0] 1772305032.927232 mapread 0x3fa0d7 thru 0x3fb4f6 (0x1420 bytes) 000387[1] 1772305032.930025 write 0x6854ba thru 0x6865a7 (0x10ee bytes) 000388[1] 1772305032.938255 mapread 0x2f3707 thru 0x2f5683 (0x1f7d bytes) 000389[0] 1772305032.951495 trunc from 0x7ae18b to 0x5dcc8f (0x1d14fb bytes) 000392[1] 1772305032.966566 mapwrite 0x705c1f thru 0x707459 (0x183b bytes) 000393[0] 1772305033.000117 write 0x09fa27 thru 0x0a11c6 (0x17a0 bytes) 000394[1] 1772305033.009766 read 0x4ec3ef thru 0x4edb00 (0x1712 bytes) 000395[1] 1772305033.038043 write 0x4e170c thru 0x4e3027 (0x191c bytes) 000397[1] 1772305033.044422 mapread 0x0ad562 thru 0x0ae355 (0xdf4 bytes) 000400[1] 1772305033.055478 mapread 0x688bab thru 0x6891ac (0x602 bytes) 000401[1] 1772305033.062155 trunc from 0x70745a to 0x915f6b (0x20eb12 bytes) 000404[1] 1772305033.068079 read 0x4708e9 thru 0x471a06 (0x111e bytes) 000405[1] 1772305033.090119 write 0x4ae4de thru 0x4afe81 (0x19a4 bytes) 000406[1] 1772305033.090654 write 0x938ce0 thru 0x93a1c1 (0x14e2 bytes) 000407[1] 1772305033.096953 write 0x07ca44 thru 0x07e865 (0x1e22 bytes) 000408[1] 1772305033.099784 trunc from 0x93a1c2 to 0x4ca8f5 (0x46f8cc bytes) 000409[1] 1772305033.109068 read 0x0b0c3a thru 0x0b15db (0x9a2 bytes) 000410[0] 1772305033.135739 write 0x583c15 thru 0x584b49 (0xf35 bytes) 000411[0] 1772305033.145852 write 0x3b274c thru 0x3b3a06 (0x12bb bytes) 000412[0] 1772305033.151954 mapread 0x2dc040 thru 0x2dde64 (0x1e25 bytes) 000414[0] 1772305033.169773 mapread 0x2f340c thru 0x2f36f0 (0x2e5 bytes) 000415[1] 1772305033.175171 trunc from 0x584b4a to 0x773698 (0x1eeb4f bytes) 000416[1] 1772305033.191666 mapwrite 0x65aa5c thru 0x65b4da (0xa7f bytes) 000417[0] 1772305033.231135 mapread 0x1e3983 thru 0x1e3fc0 (0x63e bytes) 000419[0] 1772305033.244102 trunc from 0x773699 to 0x87982b (0x106193 bytes) 000420[0] 1772305033.260407 mapwrite 0x35415f thru 0x35512e (0xfd0 bytes) 000421[1] 1772305033.292002 read 0x1f5971 thru 0x1f5d5f (0x3ef bytes) 000422[0] 1772305033.301753 read 0x608090 thru 0x6097be (0x172f bytes) 000423[1] 1772305033.314480 read 0x450124 thru 0x451cd9 (0x1bb6 bytes) 000424[0] 1772305033.346089 read 0x3d9d5b thru 0x3daaac (0xd52 bytes) 000429[0] 1772305033.352360 trunc from 0x87982c to 0x075057 (0x8047d4 bytes) 000430[0] 1772305033.372257 trunc from 0x075058 to 0x5db713 (0x5666bc bytes) 000431[0] 1772305033.383654 mapwrite 0x1d3836 thru 0x1d3fc1 (0x78c bytes) 000433[0] 1772305033.420543 mapread 0x55f5a8 thru 0x56118b (0x1be4 bytes) 000435[1] 1772305033.422563 read 0x535bb1 thru 0x5368ff (0xd4f bytes) 000436[0] 1772305033.433629 trunc from 0x5db714 to 0x8c4e7f (0x2e976c bytes) 000438[0] 1772305033.448171 read 0x76494d thru 0x765cb8 (0x136c bytes) 000439[0] 1772305033.464096 mapread 0x83da58 thru 0x83de2d (0x3d6 bytes) 000440[1] 1772305033.464663 mapread 0x73fc7b thru 0x7413dc (0x1762 bytes) 000441[0] 1772305033.476612 mapread 0x107cb4 thru 0x10816f (0x4bc bytes) 000442[1] 1772305033.479643 write 0x0a2ecf thru 0x0a37e6 (0x918 bytes) 000447[0] 1772305033.514973 write 0x40e55f thru 0x40ebb3 (0x655 bytes) 000448[0] 1772305033.526980 write 0x343f19 thru 0x3453d9 (0x14c1 bytes) 000449[0] 1772305033.537749 write 0x0b727f thru 0x0b8208 (0xf8a bytes) 000450[1] 1772305033.540577 mapwrite 0x8f36a4 thru 0x8f44df (0xe3c bytes) 000451[1] 1772305033.570054 trunc from 0x8f44e0 to 0x3bf227 (0x5352b8 bytes) 000452[0] 1772305033.579310 write 0x7b4dd0 thru 0x7b6a10 (0x1c41 bytes) 000453[0] 1772305033.586726 write 0x8c1003 thru 0x8c1d45 (0xd43 bytes) 000455[1] 1772305033.589507 mapwrite 0x04250e thru 0x043179 (0xc6c bytes) 000456[0] 1772305033.614193 read 0x216354 thru 0x217820 (0x14cd bytes) 000457[0] 1772305033.667126 read 0x814637 thru 0x814eaf (0x879 bytes) 000458[1] 1772305033.667610 mapread 0x6aceca thru 0x6ad41c (0x553 bytes) 000459[1] 1772305033.676994 trunc from 0x8c1d46 to 0x4d0987 (0x3f13be bytes) 000460[1] 1772305033.697568 read 0x1854d3 thru 0x185874 (0x3a2 bytes) 000463[1] 1772305033.739732 write 0x6d7e75 thru 0x6d9601 (0x178d bytes) 000465[1] 1772305033.746968 mapread 0x4d2494 thru 0x4d347f (0xfec bytes) 000467[1] 1772305033.773035 write 0x24b4ac thru 0x24c53f (0x1094 bytes) 000468[0] 1772305033.773599 mapread 0x1d21bc thru 0x1d397d (0x17c2 bytes) 000470[1] 1772305033.788349 trunc from 0x6d9602 to 0x114d9e (0x5c4863 bytes) 000471[0] 1772305033.803852 read 0x0c82e6 thru 0x0c9799 (0x14b4 bytes) 000473[1] 1772305033.814717 trunc from 0x114d9f to 0x1c81d1 (0xb3433 bytes) 000475[0] 1772305033.832460 write 0x7cfaaf thru 0x7d12dd (0x182f bytes) 000477[0] 1772305033.847294 mapread 0x66c75f thru 0x66e6f2 (0x1f94 bytes) 000478[1] 1772305033.856919 trunc from 0x7d12de to 0x21a5e2 (0x5b6cfb bytes) 000479[0] 1772305033.868314 mapread 0x0862fb thru 0x08752e (0x1234 bytes) 000481[1] 1772305033.877013 trunc from 0x21a5e3 to 0x5bc08a (0x3a1aa8 bytes) 000482[0] 1772305033.888524 read 0x58b531 thru 0x58ba8e (0x55e bytes) 000483[1] 1772305033.895701 read 0x573f2d thru 0x574a27 (0xafb bytes) 000484[1] 1772305033.903057 mapwrite 0x9d0a3c thru 0x9d18ec (0xeb1 bytes) 000486[1] 1772305033.956111 write 0x83f0ea thru 0x8404a8 (0x13bf bytes) 000487[0] 1772305033.956676 mapwrite 0x70b7b6 thru 0x70bd50 (0x59b bytes) 000488[0] 1772305033.984311 trunc from 0x9d18ed to 0x4c0d6b (0x510b81 bytes) 000490[0] 1772305034.001111 read 0x1ad6b3 thru 0x1af165 (0x1ab3 bytes) 000491[1] 1772305034.033708 trunc from 0x4c0d6c to 0x46918f (0x57bdc bytes) 000492[1] 1772305034.049984 trunc from 0x469190 to 0x64e0c6 (0x1e4f37 bytes) 000493[1] 1772305034.055320 read 0x214bf3 thru 0x215550 (0x95e bytes) 000494[1] 1772305034.090807 trunc from 0x64e0c7 to 0x5f4ff3 (0x590d3 bytes) 000495[0] 1772305034.096068 mapwrite 0x328956 thru 0x328b7b (0x226 bytes) 000496[0] 1772305034.119143 mapread 0x1f06f3 thru 0x1f0e09 (0x717 bytes) 000500[1] 1772305034.139523 write 0x3b6e95 thru 0x3b78ae (0xa1a bytes) All operations completed A-OK! PASS 16b (8s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 13:57:15 (1772305035) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (2s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 13:57:17 (1772305037) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0884093 s, 23.7 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0620577 s, 33.8 MB/s ldlm.namespaces.MGC192.168.202.129@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8d6402f5b800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8d6418e92800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8d6402f5b800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8d6418e92800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8d6402f5b800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8d6418e92800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 2.47368 s, 42.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.611308 s, 172 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.738931 s, 142 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.50154 s, 69.8 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 1.56086 s, 67.2 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 0.973942 s, 108 MB/s PASS 16d (11s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 13:57:28 (1772305048) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.0912515 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303571 s, 34.5 MB/s PASS 16e (3s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 13:57:31 (1772305051) TIMEOUT PASS 16f (23s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 13:57:54 (1772305074) TIMEOUT PASS 16g (22s) == sanityn test 16i: read after truncate file ============ 13:58:16 (1772305096) ------- before truncate -------- 0002700 6e65 7265 7461 726f 4420 6561 6f6d 3a6e 0002720 762f 7261 6c2f 6269 722f 676e 3a64 732f 0002740 6962 2f6e 6f6e 6f6c 6967 0a6e 0002754 ------- after truncate -------- 0001320 746e 7520 6573 2064 7962 7420 6568 7420 0001340 6f72 7375 7265 2073 6170 6b63 6761 2065 0001360 6f74 7320 6e61 0001366 PASS 16i (2s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 13:58:19 (1772305099) fail_loc=0x8000030a PASS 17 (4s) == sanityn test 18: mmap sanity check =========================================================================================== 13:58:24 (1772305104) mmap test1: basic mmap operation (PASS, 0.023757s) mmap test2: MAP_PRIVATE not write back (PASS, 0.016062s) mmap test3: concurrent mmap ops on two nodes (PASS, 8.7482s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.0923s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.02819s) mmap test6: check mmap write/read content on two nodes (PASS, 0.030619s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0185s) PASS 18 (15s) == sanityn test 19: test concurrent uncached read races ========================================================================= 13:58:39 (1772305119) oleg229-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg229-client: oleg229-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 13:58:40 (1772305120) PASS 20 (2s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 13:58:43 (1772305123) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) == sanityn test 23: others should see updated atime while another read============================================================== 13:58:45 (1772305125) now is 1772305125 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 new atime is 1772305186 PASS 23 (63s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 13:59:48 (1772305188) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 374 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 375 115.0K 1% /mnt/lustre[OST:1] filesystem_summary: 230.2K 369 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 374 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 375 115.0K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 369 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 7.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 4.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 11.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118006 374 117632 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118103 375 117728 1% /mnt/lustre2[OST:1] filesystem_summary: 235729 369 235360 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 374 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 375 115.0K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 369 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118006 374 117632 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118103 375 117728 1% /mnt/lustre[OST:1] filesystem_summary: 235729 369 235360 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118006 374 117632 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118103 375 117728 1% /mnt/lustre2[OST:1] filesystem_summary: 235729 369 235360 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 11264 7527424 1% /mnt/lustre2 PASS 24a (2s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 13:59:50 (1772305190) PASS 24b (2s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 13:59:52 (1772305192) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (2s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 13:59:55 (1772305195) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 13:59:56 (1772305196) utime: good mknod times 1772305195 <= 1772305196 <= 1772305196 for /mnt/lustre/f26a utime: good mknod times 1772305195 <= 1772305196 <= 1772305196 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (3s) == sanityn test 26b: sync mtime between ost and mds ====== 13:59:59 (1772305199) PASS 26b (4s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 14:00:04 (1772305204) 0 PASS 30 (4s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 14:00:08 (1772305208) fail_loc=0x314 PASS 31a (3s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 14:00:11 (1772305211) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg229-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.202.129@tcp:/lustre 7542784 12288 7524352 1% /mnt/lustre2 PASS 31b (12s) == sanityn test 31r: open-rename(replace) race =========== 14:00:23 (1772305223) -rw-r--r-- 1 root root 0 Feb 28 14:00 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (5s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 14:00:29 (1772305229) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 14:00:30 (1772305230) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 14:00:31 (1772305231) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 14:00:32 (1772305232) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 14:00:33 (1772305233) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg229-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8d6402f5b800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8d6418e92800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8d6402f5b800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8d6418e92800.ost_server_uuid in FULL state after 0 sec oleg229-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8d6402f5b800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8d6418e92800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8d6402f5b800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8d6418e92800.ost_server_uuid in IDLE state after 0 sec oleg229-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8d6402f5b800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8d6418e92800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8d6402f5b800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8d6418e92800.ost_server_uuid in FULL state after 0 sec oleg229-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8d6402f5b800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8d6418e92800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8d6402f5b800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8d6418e92800.ost_server_uuid in IDLE state after 0 sec oleg229-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8d6402f5b800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8d6418e92800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8d6402f5b800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8d6418e92800.ost_server_uuid in IDLE state after 0 sec oleg229-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8d6402f5b800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8d6418e92800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8d6402f5b800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8d6418e92800.ost_server_uuid in IDLE state after 0 sec PASS 34 (80s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 14:01:53 (1772305313) fail_loc=0x80000317 Race attempt 0 Wait for 44625 44756 for 60 sec... total: 4000 open/close in 7.90 seconds: 506.13 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.202.129@tcp:/lustre 7.2G 14M 7.2G 1% /mnt/lustre 192.168.202.129@tcp:/lustre 7.2G 14M 7.2G 1% /mnt/lustre2 PASS 35 (64s) == sanityn test 36: handle ESTALE/open-unlink correctly == 14:02:57 (1772305377) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg229-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.380785 s, 138 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.396112 s, 132 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.359662 s, 146 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.374647 s, 140 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.382302 s, 137 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.361422 s, 145 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.380745 s, 138 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.374375 s, 140 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.334393 s, 157 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.403335 s, 130 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB, 50 MiB) copied, 0.334442 s, 157 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7524352) after_dd(7473152) after(7524352) PASS 36 (245s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 14:07:02 (1772305622) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 - create 9478 (time 1772305634.00 total 10.00 last 947.74) total: 10000 create in 10.53 seconds: 949.33 ops/second PASS 37 (14s) == sanityn test 39a: test from 11063 ============================================================================================ 14:07:16 (1772305636) repeat after cancel_lru_locks PASS 39a (3s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 14:07:19 (1772305639) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 14:07:22 (1772305642) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 14:07:26 (1772305646) fail_loc=0x411 fail_loc=0 PASS 39d (2s) == sanityn test 40a: pdirops: create vs others ======================================================================== 14:07:28 (1772305648) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (8s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 14:07:36 (1772305656) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (9s) == sanityn test 40c: pdirops: link and others ======================================================================== 14:07:45 (1772305665) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (8s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 14:07:53 (1772305673) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (9s) == sanityn test 40e: pdirops: rename and others ======================================================================== 14:08:02 (1772305682) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (8s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 14:08:10 (1772305690) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (6s) == sanityn test 41b: pdirops: create vs create ======================================================================== 14:08:16 (1772305696) open: File exists Conflict PASS 41b (7s) == sanityn test 41c: pdirops: create vs link ======================================================================== 14:08:23 (1772305703) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (5s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 14:08:29 (1772305709) Conflict PASS 41d (5s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 14:08:34 (1772305714) Conflict PASS 41e (5s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 14:08:40 (1772305720) Conflict PASS 41f (6s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 14:08:46 (1772305726) Conflict PASS 41g (7s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 14:08:54 (1772305734) Conflict PASS 41h (7s) == sanityn test 41i: reint_open: create vs create ======== 14:09:00 (1772305740) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (744s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 14:21:24 (1772306484) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (6s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 14:21:30 (1772306490) open: File exists Conflict PASS 42b (6s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 14:21:36 (1772306496) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (5s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 14:21:41 (1772306501) Conflict PASS 42d (6s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 14:21:47 (1772306507) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (6s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 14:21:53 (1772306513) Conflict PASS 42f (6s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 14:21:59 (1772306519) Conflict PASS 42g (6s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 14:22:05 (1772306525) Conflict PASS 42h (6s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 14:22:11 (1772306531) PASS 43a (15s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 14:22:26 (1772306546) Conflict PASS 43b (5s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 14:22:31 (1772306551) Conflict PASS 43c (5s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 14:22:36 (1772306556) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (6s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 14:22:42 (1772306562) Conflict PASS 43e (6s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 14:22:48 (1772306568) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (6s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 14:22:54 (1772306574) stat: cannot statx '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (6s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 14:23:00 (1772306580) Conflict PASS 43h (6s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 14:23:06 (1772306586) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 14:23:07 (1772306587) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (49s) == sanityn test 43k: unlink vs create ==================== 14:23:56 (1772306636) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (451s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 14:31:27 (1772307087) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Sat Feb 28 14:31:30 EST 2026 PASS 44a (5s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 14:31:32 (1772307092) open: File exists Conflict PASS 44b (5s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 14:31:37 (1772307097) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (5s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 14:31:42 (1772307102) Conflict PASS 44d (5s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 14:31:47 (1772307107) Conflict PASS 44e (5s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 14:31:52 (1772307112) Conflict PASS 44f (5s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 14:31:57 (1772307117) Conflict PASS 44g (6s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 14:32:03 (1772307123) Conflict PASS 44h (5s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 14:32:08 (1772307128) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 14:32:09 (1772307129) PASS 45a (23s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 14:32:32 (1772307152) Conflict PASS 45b (5s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 14:32:37 (1772307157) Conflict PASS 45c (5s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 14:32:42 (1772307162) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (5s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 14:32:47 (1772307167) Conflict PASS 45e (5s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 14:32:52 (1772307172) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (5s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 14:32:57 (1772307177) stat: cannot statx '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (4s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 14:33:01 (1772307181) Conflict PASS 45h (5s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 14:33:06 (1772307186) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 14:33:07 (1772307187) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (463s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 14:40:50 (1772307650) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (6s) == sanityn test 46b: pdirops: link vs create ======================================================================== 14:40:56 (1772307656) open: File exists Conflict PASS 46b (6s) == sanityn test 46c: pdirops: link vs link ======================================================================== 14:41:03 (1772307663) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (6s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 14:41:09 (1772307669) Conflict PASS 46d (7s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 14:41:16 (1772307676) Conflict PASS 46e (6s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 14:41:22 (1772307682) Conflict PASS 46f (7s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 14:41:29 (1772307689) Conflict PASS 46g (6s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 14:41:35 (1772307695) Conflict PASS 46h (6s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 14:41:41 (1772307701) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 14:41:43 (1772307703) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 14:41:44 (1772307704) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 14:41:46 (1772307706) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (2s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 14:41:47 (1772307707) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (2s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 14:41:49 (1772307709) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 14:41:50 (1772307710) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (2s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 14:41:52 (1772307712) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (2s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 14:41:54 (1772307714) 10+0 records in 10+0 records out 10240 bytes (10 kB, 10 KiB) copied, 0.00839808 s, 1.2 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 14:42:01 (1772307721) 0+1 records in 0+1 records out 158 bytes copied, 0.00594822 s, 26.6 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (242749) - No such process PASS 51a (5s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 14:42:06 (1772307726) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00382304 s, 268 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236069 s, 44.4 MB/s 3145728 PASS 51b (15s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 14:42:21 (1772307741) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00593381 s, 173 kB/s PASS 51c (8s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 14:42:29 (1772307749) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223029 s, 47.0 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (4s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 14:42:33 (1772307753) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2435 0x983 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2435 0x983 0 read lease(1) has applied. PASS 51e (5s) == sanityn test 54: rename locking ======================= 14:42:38 (1772307758) llite.lustre-ffff8d6402f5b800.xattr_cache=1 llite.lustre-ffff8d6418e92800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504962 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-02-28 14:42:38.000000000 -0500 Modify: 2026-02-28 14:42:38.000000000 -0500 Change: 2026-02-28 14:42:38.000000000 -0500 Birth: 2026-02-28 14:42:38.000000000 -0500 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504965 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-02-28 14:42:44.000000000 -0500 Modify: 2026-02-28 14:42:44.000000000 -0500 Change: 2026-02-28 14:42:44.000000000 -0500 Birth: 2026-02-28 14:42:44.000000000 -0500 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: /mnt/lustre/d1/d2 Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-02-28 14:42:49.000000000 -0500 Modify: 2026-02-28 14:42:49.000000000 -0500 Change: 2026-02-28 14:42:49.000000000 -0500 Birth: 2026-02-28 14:42:49.000000000 -0500 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504967 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-02-28 14:42:49.000000000 -0500 Modify: 2026-02-28 14:42:49.000000000 -0500 Change: 2026-02-28 14:42:49.000000000 -0500 Birth: 2026-02-28 14:42:49.000000000 -0500 ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: /mnt/lustre/d1 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504970 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-02-28 14:42:55.000000000 -0500 Modify: 2026-02-28 14:42:55.000000000 -0500 Change: 2026-02-28 14:42:55.000000000 -0500 Birth: 2026-02-28 14:42:55.000000000 -0500 File: /mnt/lustre/d1/d2 Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-02-28 14:42:55.000000000 -0500 Modify: 2026-02-28 14:42:55.000000000 -0500 Change: 2026-02-28 14:42:55.000000000 -0500 Birth: 2026-02-28 14:42:55.000000000 -0500 PASS 54 (25s) == sanityn test 55a: rename vs unlink target dir ========= 14:43:03 (1772307783) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (8s) == sanityn test 55b: rename vs unlink source dir ========= 14:43:11 (1772307791) fail_loc=0x80000156 mv: cannot stat '/mnt/lustre/d55b.sanityn/d1/d2': No such file or directory PASS 55b (8s) == sanityn test 55c: rename vs unlink orphan target dir == 14:43:19 (1772307799) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (13s) == sanityn test 55d: rename file vs link ================= 14:43:32 (1772307812) fail_loc=0x155 mv: '/mnt/lustre/d55d.sanityn/f1' and '/mnt/lustre/d55d.sanityn/d55d.sanityn/f1' are the same file PASS 55d (10s) == sanityn test 60: Verify data_version behaviour ======== 14:43:42 (1772307822) PASS 60 (3s) == sanityn test 70a: cd directory && rm directory ======== 14:43:45 (1772307825) PASS 70a (3s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 14:43:48 (1772307828) PASS 70b (3s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 14:43:51 (1772307831) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00241236 s, 27.2 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (2s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 14:43:53 (1772307833) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.00365022 s, 18.0 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 14:43:54 (1772307834) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (2s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 14:43:56 (1772307836) 1+0 records in 1+0 records out 65536 bytes (66 kB, 64 KiB) copied, 0.0057855 s, 11.3 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 14:43:58 (1772307838) llite.lustre-ffff8d6402f5b800.xattr_cache=1 llite.lustre-ffff8d6418e92800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (3s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 14:44:00 (1772307840) llite.lustre-ffff8d6402f5b800.xattr_cache=1 llite.lustre-ffff8d6418e92800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (3s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 14:44:04 (1772307844) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00277598 s, 369 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00291447 s, 351 kB/s 252777: get lock2 252779: get lock1 252777: done 252777: get lock1 252779: done 252779: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 252779: cannot get lock2: Resource deadlock avoided 252779: exit rc=0 fcntl cmd 7 failed: Resource deadlock avoided 252777: cannot get lock1: Resource deadlock avoided 252777: done 252777: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 14:44:10 (1772307850) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0198146 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891856 s, 118 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 253352 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 14:44:17 (1772307857) opening files: ........ closing files: ........ PASS 76 (25s) == sanityn test 77a: check FIFO NRS policy =============== 14:44:42 (1772307882) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.112722 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332329 s, 31.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0465582 s, 22.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0547858 s, 19.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0435686 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0587175 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0723906 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0496184 s, 21.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0841744 s, 12.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0698166 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0860377 s, 12.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0756354 s, 13.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0938116 s, 11.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0785453 s, 13.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.090589 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0752165 s, 13.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8d6418e92800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8d6418e92800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0956868 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123867 s, 84.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209161 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161199 s, 65.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175667 s, 59.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236081 s, 44.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025669 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240123 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231299 s, 45.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155258 s, 67.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139316 s, 75.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217708 s, 48.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230023 s, 45.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0124672 s, 84.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00960867 s, 109 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087537 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132575 s, 79.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8d6418e92800.lock_unused_count=1 removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg229-client.virtnet' removed directory '/mnt/lustre/d77a.sanityn' PASS 77a (3s) == sanityn test 77b: check CRR-N NRS policy ============== 14:44:45 (1772307885) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.156806 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0497927 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0554955 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0635525 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0532957 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0717032 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0564253 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0684759 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0638184 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0706412 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0944929 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0907024 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.07742 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0692438 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0794652 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0886729 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0776544 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171547 s, 61.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00413019 s, 254 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087198 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0129109 s, 81.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186956 s, 56.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130153 s, 80.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169659 s, 61.8 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0001-osc-ffff8d6418e92800.lock_unused_count=1 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235239 s, 44.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182425 s, 57.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0060861 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0091854 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00324424 s, 323 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973027 s, 108 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679943 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155749 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700172 s, 150 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg229-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.16664 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223162 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0364045 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0404299 s, 25.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0592581 s, 17.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0489088 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0744964 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0841842 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0839596 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0601548 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.094431 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0628228 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0757608 s, 13.8 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8d6418e92800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.091368 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0990559 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0826643 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0998018 s, 10.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00943283 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177004 s, 59.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140507 s, 74.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227918 s, 46.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244789 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310372 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146351 s, 71.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174482 s, 60.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019507 s, 53.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299025 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235793 s, 44.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814752 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273763 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0340585 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0329711 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00489488 s, 214 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8d6418e92800.lock_unused_count=1 removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg229-client.virtnet' removed directory '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (5s) == sanityn test 77c: check ORR NRS policy ================ 14:44:50 (1772307890) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.136767 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243901 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283064 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0414066 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0665614 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0816636 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0675502 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0887973 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0777971 s, 13.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8d6418e92800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff8d6418e92800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0710565 s, 14.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0997982 s, 10.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0938223 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0971382 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0951098 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109753 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118301 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100271 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117682 s, 89.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206493 s, 50.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203283 s, 51.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128549 s, 81.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179687 s, 58.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104803 s, 100 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125685 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190898 s, 54.9 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00368262 s, 285 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283098 s, 37.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281762 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0126328 s, 83.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132757 s, 79.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118042 s, 88.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00372455 s, 282 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102089 s, 103 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8d6418e92800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8d6418e92800.lock_unused_count=2 removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg229-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.158774 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297872 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0371246 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0586915 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0501292 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0655464 s, 16.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8d6418e92800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.048945 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0795098 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0756335 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101556 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118888 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106729 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102149 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11631 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119028 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0123983 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134789 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255428 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022457 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116036 s, 9.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241503 s, 43.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263576 s, 39.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200846 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356885 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289671 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186999 s, 56.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121478 s, 86.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00397021 s, 264 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00402056 s, 261 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0039824 s, 263 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00452649 s, 232 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8d6418e92800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff8d6418e92800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848729 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.148705 s, 7.1 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg229-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.117466 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305183 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0399101 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0335196 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0346608 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0487727 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0607651 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.050097 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0557623 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0610285 s, 17.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8d6418e92800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0872077 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.09587 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.110298 s, 9.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102116 s, 10.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108982 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10004 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0989841 s, 10.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8d6418e92800.lock_unused_count=5 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012204 s, 85.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019095 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190039 s, 55.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154352 s, 67.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191822 s, 54.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00337104 s, 311 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146058 s, 71.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168025 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168081 s, 62.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186856 s, 56.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218346 s, 48.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00403106 s, 260 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0092028 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255039 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666752 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841061 s, 125 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg229-client.virtnet' removed directory '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (7s) == sanityn test 77d: check TRR nrs policy ================ 14:44:57 (1772307897) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.159679 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0338103 s, 31.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0617025 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0686924 s, 15.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0559768 s, 18.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0650449 s, 16.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0507905 s, 20.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0511657 s, 20.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0592773 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0922776 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0857553 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0954257 s, 11.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0960101 s, 10.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0852487 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0922296 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0769851 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100922 s, 10.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178917 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188769 s, 55.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106254 s, 98.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0122629 s, 85.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00439076 s, 239 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155569 s, 67.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221366 s, 47.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145045 s, 72.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146695 s, 71.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219476 s, 47.8 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00387053 s, 271 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196661 s, 53.3 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196165 s, 53.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00388404 s, 270 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019854 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00161022 s, 651 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg229-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.12574 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0399451 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0871835 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0884786 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.100883 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0956007 s, 11.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8d6418e92800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119519 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139457 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.144308 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128016 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.157374 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134141 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131667 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128665 s, 8.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8d6418e92800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137639 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149989 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.154429 s, 6.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107076 s, 97.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023968 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209553 s, 50.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223298 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326247 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.043334 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260204 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0311836 s, 33.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314356 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300915 s, 34.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336728 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292377 s, 35.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0347655 s, 30.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0322374 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0373742 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0340584 s, 30.8 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg229-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.1052 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222843 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0425923 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0481435 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.083307 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0790856 s, 13.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0874953 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0945996 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0784863 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.084092 s, 12.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0779236 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0766124 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111492 s, 9.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107006 s, 9.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0880191 s, 11.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0938201 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823869 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141203 s, 74.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178796 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140069 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156986 s, 66.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174595 s, 60.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204741 s, 51.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138534 s, 75.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00621816 s, 169 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025289 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285952 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207023 s, 50.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8d6418e92800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8d6418e92800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136875 s, 76.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295756 s, 35.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.012257 s, 85.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00394268 s, 266 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304701 s, 34.4 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg229-client.virtnet' removed directory '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (5s) == sanityn test 77e: check TBF NID nrs policy ============ 14:45:02 (1772307902) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.202.29@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.188472 s, 89.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0369554 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0530961 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0564143 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0580338 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0664945 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0464058 s, 22.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0936569 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0868794 s, 12.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0883064 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0713371 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.078208 s, 13.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0755129 s, 13.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0660169 s, 15.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.076263 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0854267 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0920385 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196571 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224566 s, 46.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201065 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207451 s, 50.5 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191421 s, 54.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019708 s, 53.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00437586 s, 240 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084931 s, 123 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011147 s, 94.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272966 s, 38.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00557929 s, 188 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270033 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248 s, 42.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0061893 s, 169 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809006 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00524852 s, 200 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg229-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.1368 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256513 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0485394 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0687039 s, 15.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0730077 s, 14.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0676341 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0724774 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0618712 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0477675 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0758614 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0740062 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0853012 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0685606 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0717624 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.083036 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0817551 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0931009 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240571 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0316565 s, 33.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018862 s, 55.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140169 s, 74.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185079 s, 56.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262599 s, 39.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253167 s, 41.4 MB/s1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324128 s, 32.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0351271 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00417341 s, 251 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0379565 s, 27.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0314638 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00372294 s, 282 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643633 s, 163 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309909 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00417066 s, 251 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg229-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.152714 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0404301 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0504797 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.058672 s, 17.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0745256 s, 14.1 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.052021 s, 20.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0782105 s, 13.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0880298 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0822652 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0763887 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0768412 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0710768 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0641826 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0573775 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0548959 s, 19.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.063601 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.068071 s, 15.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166209 s, 63.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115846 s, 90.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171046 s, 61.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143376 s, 73.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183783 s, 57.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0147744 s, 71.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237714 s, 44.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157941 s, 66.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823385 s, 127 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245598 s, 42.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230444 s, 45.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189949 s, 55.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704412 s, 149 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261348 s, 40.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811304 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0026733 s, 392 MB/sldlm.namespaces.lustre-OST0001-osc-ffff8d6418e92800.lock_unused_count=1 removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg229-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.126702 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028495 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0518309 s, 20.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0645168 s, 16.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0564071 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0694915 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.078349 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0717779 s, 14.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0722593 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0807978 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.058422 s, 17.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0701104 s, 15.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0803818 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0941988 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0819947 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0861293 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0806404 s, 13.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176876 s, 59.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164309 s, 63.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205622 s, 51.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142052 s, 73.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175212 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653707 s, 160 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101216 s, 104 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00474272 s, 221 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257919 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230099 s, 45.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024114 s, 43.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00329074 s, 319 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159967 s, 65.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769053 s, 136 MB/sldlm.namespaces.lustre-OST0001-osc-ffff8d6418e92800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212151 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130153 s, 80.6 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg229-client.virtnet' removed directory '/mnt/lustre/d77e.sanityn' PASS 77e (10s) == sanityn test 77f: check TBF JobID nrs policy ========== 14:45:12 (1772307912) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.183414 s, 91.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0488535 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0401904 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0447202 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0611989 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0496716 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0538695 s, 19.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.071877 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0799161 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0958507 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0810053 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0945435 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0998177 s, 10.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0779197 s, 13.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106876 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102513 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0925247 s, 11.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0163306 s, 64.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184626 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022989 s, 45.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172328 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229692 s, 45.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291162 s, 36.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241825 s, 43.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154432 s, 67.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0318009 s, 33.0 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192693 s, 54.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189419 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694405 s, 151 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0392428 s, 26.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372668 s, 28.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372945 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989523 s, 106 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg229-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.143489 s, 117 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0397124 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0673655 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0505934 s, 20.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120339 s, 8.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125811 s, 8.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120617 s, 8.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106385 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123654 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128935 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11876 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.153062 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128107 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126122 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122036 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.158065 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.152811 s, 6.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8d6418e92800.lock_unused_count=4 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186704 s, 56.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206708 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294428 s, 35.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293029 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0141163 s, 74.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268092 s, 39.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229161 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294547 s, 35.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169028 s, 62.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015046 s, 69.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0037724 s, 278 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116003 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030058 s, 34.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213055 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.004003 s, 262 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0145409 s, 72.1 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg229-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.14667 s, 114 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0511177 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0744083 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0761931 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0988324 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0947422 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105355 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117465 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127746 s, 8.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r]1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103595 s, 10.1 MB/s execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12529 s, 8.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.147812 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116002 s, 9.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122083 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.139213 s, 7.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8d6418e92800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.141518 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146557 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195573 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106956 s, 98.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224266 s, 46.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00446656 s, 235 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0345206 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188892 s, 55.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223541 s, 46.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697028 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110129 s, 95.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00560388 s, 187 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00539317 s, 194 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0330282 s, 31.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869774 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00905663 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00453038 s, 231 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638091 s, 164 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg229-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.168809 s, 99.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0648657 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0831898 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109265 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103809 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.116864 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120678 s, 8.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114079 s, 9.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12826 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114324 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117633 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125547 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11313 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113514 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133826 s, 7.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122164 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130177 s, 8.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0132444 s, 79.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020093 s, 52.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021715 s, 48.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242098 s, 43.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227432 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188402 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784493 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185584 s, 56.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287612 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0324037 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272709 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229038 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182795 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208374 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00562129 s, 187 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015672 s, 66.9 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg229-client.virtnet' removed directory '/mnt/lustre/d77f.sanityn' PASS 77f (10s) == sanityn test 77g: Change TBF type directly ============ 14:45:22 (1772307922) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (5s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 14:45:27 (1772307927) oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg229-client: oleg229-server: ssh exited with exit code 2 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg229-client: oleg229-server: ssh exited with exit code 12 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg229-client: oleg229-server: ssh exited with exit code 12 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg229-client: oleg229-server: ssh exited with exit code 12 PASS 77h (6s) == sanityn test 77i: Change rank of TBF rule ============= 14:45:33 (1772307933) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (9s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 14:45:42 (1772307942) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.106188 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0357007 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0534569 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0826973 s, 12.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.085874 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0872357 s, 12.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0676059 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0609768 s, 17.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0891307 s, 11.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0767563 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0756589 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0737034 s, 14.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0924722 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0831188 s, 12.6 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0882001 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0894412 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0861405 s, 12.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193135 s, 54.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146597 s, 71.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232612 s, 45.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171382 s, 61.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241407 s, 43.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00472592 s, 222 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249697 s, 42.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221464 s, 47.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00282529 s, 371 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226093 s, 46.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128328 s, 81.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226656 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246343 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288297 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00951847 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138591 s, 75.7 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg229-client.virtnet' removed directory '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.9603 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.448 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (53s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 14:46:35 (1772307995) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.681074 s, 24.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11995 s, 8.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.103848 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0984669 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109191 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118608 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111699 s, 9.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117175 s, 8.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.122656 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.322152 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.343549 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.344246 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.362474 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.391307 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.361857 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.385886 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.363195 s, 2.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.125126 s, 8.4 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130584 s, 8.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.127452 s, 8.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134037 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131095 s, 8.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.137153 s, 7.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.128136 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130506 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.12904 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131537 s, 8.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.132349 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.136158 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130744 s, 8.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.149445 s, 7.0 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131231 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.140838 s, 7.4 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg229-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.8116 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9145 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.676407 s, 24.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160111 s, 6.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.145954 s, 7.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.15647 s, 6.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.17408 s, 6.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150308 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.173042 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.168994 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.160487 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.359116 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.382825 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.378505 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.400327 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.386262 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.405332 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.401992 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.396306 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10968 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.107799 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.10829 s, 9.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106737 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111657 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114584 s, 9.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119848 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.108106 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113399 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119164 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.129161 s, 8.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114548 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11495 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.119269 s, 8.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.123053 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.118818 s, 8.8 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg229-client.virtnet' removed directory '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.7104 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.9146 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (136s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 14:48:51 (1772308131) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.227553 s, 73.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0441819 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0484457 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0356811 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0450657 s, 23.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.057726 s, 18.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0579697 s, 18.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0651159 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0789792 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0793774 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0719453 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104038 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0944651 s, 11.1 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0834729 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0846647 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0891269 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0935647 s, 11.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189867 s, 55.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205535 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189807 s, 55.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201443 s, 52.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201689 s, 52.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021191 s, 49.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219017 s, 47.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00943999 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192933 s, 54.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00464313 s, 226 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115881 s, 90.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019461 s, 53.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024386 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251828 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00293383 s, 357 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00271089 s, 387 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg229-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.02204 s, 20.9 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.7433 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.202.29@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.202.29@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.158113 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0371188 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0523843 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0651814 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0746515 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0939043 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0617379 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0893557 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0856911 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0815507 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0823414 s, 12.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13822 s, 7.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130565 s, 8.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.245019 s, 4.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.223772 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.325176 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.329667 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191098 s, 54.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192567 s, 54.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.015589 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0139329 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222251 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134983 s, 77.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216432 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00400556 s, 262 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237408 s, 44.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275159 s, 38.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155965 s, 67.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857415 s, 122 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752303 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00378428 s, 277 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272627 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00533415 s, 197 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 5.01715 s, 20.9 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74384 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.202.29@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.222537 s, 75.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0473294 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.078267 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0665596 s, 15.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0735435 s, 14.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0635783 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0983831 s, 10.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0837298 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0971987 s, 10.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0857145 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0816785 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0890951 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0999204 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0855581 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0764422 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0889323 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.106108 s, 9.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278968 s, 37.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237347 s, 44.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256174 s, 40.9 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255087 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239805 s, 43.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233556 s, 44.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197269 s, 53.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0162225 s, 64.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192441 s, 54.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252943 s, 41.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00423002 s, 248 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068085 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197805 s, 53.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257858 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149225 s, 70.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00404371 s, 259 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg229-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.93086 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.89466 s, 21.4 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.254651 s, 65.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234032 s, 44.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0700755 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0644689 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0915554 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0894206 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0791843 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0846263 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0594973 s, 17.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146236 s, 7.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150669 s, 7.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.294174 s, 3.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.272655 s, 3.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.331584 s, 3.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.32686 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.319389 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.341665 s, 3.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201779 s, 52.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192434 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142063 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169689 s, 61.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0115067 s, 91.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022715 s, 46.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242324 s, 43.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243681 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017059 s, 61.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233131 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00411664 s, 255 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106549 s, 98.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244129 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00377374 s, 278 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257625 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0043747 s, 240 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.77267 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.73483 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.91005 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.72746 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.448932 s, 37.4 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.057278 s, 18.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.113932 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.120534 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.130859 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.111645 s, 9.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.143823 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.133315 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.121139 s, 8.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.338952 s, 3.1 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.331475 s, 3.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.332156 s, 3.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.344299 s, 3.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.36117 s, 2.9 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.349166 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.343334 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.349046 s, 3.0 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143808 s, 72.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174906 s, 60.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0152854 s, 68.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243669 s, 43.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242262 s, 43.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216204 s, 48.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100939 s, 104 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00583446 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190531 s, 55.0 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00521221 s, 201 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109028 s, 96.2 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221249 s, 47.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130079 s, 80.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248006 s, 42.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728138 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087009 s, 121 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.6687 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 19.4314 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.167505 s, 100 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0393968 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.047731 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0546498 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0843168 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.105184 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0943044 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.087733 s, 12.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0966811 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0922699 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0913273 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101128 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.079124 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0913674 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.114325 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.104295 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.117453 s, 8.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0146539 s, 71.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254303 s, 41.2 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269876 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0847294 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020605 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315344 s, 33.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0319683 s, 32.8 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243643 s, 43.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246939 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258501 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0331291 s, 31.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0961837 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0310662 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0413523 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0347755 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0372954 s, 28.1 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.95275 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74141 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.2657 s, 63.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0592227 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0811079 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.11484 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.112007 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.126272 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.131827 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.134226 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.146242 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.13368 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.150238 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.274046 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.269039 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.316903 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.302254 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.373405 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.386376 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190804 s, 55.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273155 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02627 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236266 s, 44.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231434 s, 45.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00432342 s, 243 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201188 s, 52.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183647 s, 57.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279805 s, 37.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305969 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00528507 s, 198 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00972345 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248558 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860951 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171607 s, 61.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112261 s, 93.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg229-client.virtnet' removed directory '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.79168 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74576 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.79677 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 9.74005 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (366s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 14:54:57 (1772308497) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (4s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 14:55:01 (1772308501) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg229-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 3.10163 s, 338 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg229-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 40.015 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (52s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 14:55:53 (1772308553) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.145913 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0344045 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0791967 s, 13.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0897584 s, 11.7 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102827 s, 10.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0836265 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.099671 s, 10.5 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0979422 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.102221 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.088268 s, 11.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.101578 s, 10.3 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0875054 s, 12.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0917195 s, 11.4 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0973647 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0997364 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0884677 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.109945 s, 9.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8d6418e92800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8d6418e92800.lock_unused_count=2 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206227 s, 50.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190764 s, 55.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176369 s, 59.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193382 s, 54.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215093 s, 48.7 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0155097 s, 67.6 MB/s1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017717 s, 59.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023287 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159999 s, 65.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011819 s, 88.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176915 s, 59.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288886 s, 36.3 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00596764 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025013 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219606 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260205 s, 40.3 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg229-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.96926 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.971 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB, 16 MiB) copied, 0.195985 s, 85.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0483314 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0597817 s, 17.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0611717 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0639673 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0639007 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0642797 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0554091 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0768606 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0644352 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0722694 s, 14.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.156937 s, 6.7 MB/s1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.135153 s, 7.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.226781 s, 4.6 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.234878 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.348611 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.341005 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0143715 s, 73.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212278 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233265 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169662 s, 61.8 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213079 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0158793 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248785 s, 42.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232192 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253571 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207307 s, 50.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205927 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212949 s, 49.2 MB/s 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00435091 s, 241 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260505 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228682 s, 45.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251722 s, 41.7 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg229-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg229-client.virtnet' removed directory '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.91125 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB, 100 MiB) copied, 4.97528 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (66s) == sanityn test 77o: Changing rank should not panic ====== 14:56:59 (1772308619) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (5s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 14:57:04 (1772308624) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (46s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 14:57:50 (1772308670) mds.MDS.mdt.nrs_policies=tbf oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg229-client: oleg229-server: ssh exited with exit code 36 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg229-client: oleg229-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (16s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 14:58:06 (1772308686) oleg229-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device pdsh@oleg229-client: oleg229-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (4s) == sanityn test 79: xattr: intent error ================== 14:58:10 (1772308690) File: /mnt/lustre/d79.sanityn Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505430 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-02-28 14:58:11.000000000 -0500 Modify: 2026-02-28 14:58:11.000000000 -0500 Change: 2026-02-28 14:58:11.000000000 -0500 Birth: 2026-02-28 14:58:11.000000000 -0500 fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (13s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 14:58:23 (1772308703) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 14:58:24 (1772308704) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 14:58:25 (1772308705) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 14:58:26 (1772308706) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (2s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 14:58:28 (1772308708) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 14:58:29 (1772308709) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7142 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7142 PASS 82 (3s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 14:58:32 (1772308712) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 14:58:33 (1772308713) fail_loc=0x8000060b PASS 84 (9s) == sanityn test 90: open/create and unlink striped directory ========================================================== 14:58:42 (1772308722) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (2s) == sanityn test 91: chmod and unlink striped directory === 14:58:44 (1772308724) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 14:58:46 (1772308726) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 14:58:47 (1772308727) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2630 0xa46 0 0 2644 0xa54 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2645 0xa55 0 1 2631 0xa47 0 rc1=0 and rc2=0 PASS 93 (11s) == sanityn test 94: signal vs CP callback race =========== 14:58:58 (1772308738) osc.lustre-OST0000-osc-ffff8d6402f5b800.idle_timeout=0 osc.lustre-OST0000-osc-ffff8d6418e92800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8d6402f5b800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8d6418e92800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0672811 s, 122 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 3.08928 s, 1.3 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 284118 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0189552 s, 216 kB/s osc.lustre-OST0000-osc-ffff8d6402f5b800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8d6418e92800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8d6402f5b800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8d6418e92800.idle_timeout=debug PASS 94 (11s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 14:59:09 (1772308749) llite.lustre-ffff8d6402f5b800.fast_read=0 llite.lustre-ffff8d6418e92800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB, 12 KiB) copied, 0.00334984 s, 3.7 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8d6402f5b800.fast_read=1 llite.lustre-ffff8d6418e92800.fast_read=1 PASS 95b (9s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 14:59:18 (1772308758) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (2s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 14:59:20 (1772308760) 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00133871 s, 3.1 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (3s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 14:59:23 (1772308763) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0243458 s, 86.1 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 14:59:25 (1772308765) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0261587 s, 80.2 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 14:59:27 (1772308767) 1+0 records in 1+0 records out 12000 bytes (12 kB, 12 KiB) copied, 0.00216172 s, 5.6 MB/s PASS 100e (3s) == sanityn test 101a: Discard DoM data on unlink ========= 14:59:30 (1772308770) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000735898 s, 5.6 MB/s PASS 101a (3s) == sanityn test 101b: Discard DoM data on rename ========= 14:59:33 (1772308773) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.000592178 s, 6.9 MB/s PASS 101b (3s) == sanityn test 101c: Discard DoM data on close-unlink === 14:59:36 (1772308776) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00069147 s, 5.9 MB/s PASS 101c (3s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 14:59:40 (1772308780) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1772308781 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1772308784 Finishing test test23 at 1772308784 PASS 103 (8s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 14:59:48 (1772308788) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (2s) == sanityn test 105: Glimpse and lock cancel race ======== 14:59:50 (1772308790) fail_loc=0x416 PASS 105 (48s) == sanityn test 106a: Verify the btime via statx() ======= 15:00:38 (1772308838) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 15:00:39 (1772308839) 1+0 records in 1+0 records out 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00323374 s, 317 kB/s File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505458 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-02-28 15:00:40.000000000 -0500 Modify: 2026-02-28 15:00:40.000000000 -0500 Change: 2026-02-28 15:00:40.000000000 -0500 Birth: 2026-02-28 15:00:40.000000000 -0500 /mnt/lustre/f106b.sanityn 144115205272505458 -rw-r--r-- 1 0 0 1772308840 1772308840 1772308840 File: /mnt/lustre/f106b.sanityn Size: 1024 Blocks: 1 IO Block: 4194304 regular file Device: 2c54f966h/743766374d Inode: 144115205272505458 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-02-28 15:00:40.000000000 -0500 Modify: 2026-02-28 15:00:40.000000000 -0500 Change: 2026-02-28 15:00:40.000000000 -0500 Birth: 2026-02-28 15:00:40.000000000 -0500 1772308840 1024 1 PASS 106b (3s) == sanityn test 106c: Verify statx attributes mask ======= 15:00:42 (1772308842) PASS 106c (2s) == sanityn test 107a: Basic grouplock conflict =========== 15:00:44 (1772308844) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.0854202 s, 123 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2653 0xa5d 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 15:00:48 (1772308848) 10+0 records in 10+0 records out 10485760 bytes (10 MB, 10 MiB) copied, 0.0834774 s, 126 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2636 0xa4c 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7142 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) == sanityn test 108a: lseek: parallel updates ============ 15:00:56 (1772308856) 1+0 records in 1+0 records out 10000 bytes (10 kB, 9.8 KiB) copied, 0.00199327 s, 5.0 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB, 8.0 MiB) copied, 4.37953 s, 1.9 MB/s PASS 108a (7s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 15:01:03 (1772308863) 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 192.168.202.129@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 192.168.202.129@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,encrypt 0 0 Stopping client oleg229-client.virtnet /mnt/lustre (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 (opts:) /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 99: echo: write error: Device or resource busy LNET ready to unload /home/green/git/lustre-release/lustre/scripts/lustre_rmmod: line 141: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg229-server' oleg229-server: oleg229-server.virtnet: executing load_modules_local oleg229-server: Loading modules from /home/green/git/lustre-release/lustre oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 712: echo: write error: Device or resource busy oleg229-server: /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 713: echo: write error: Device or resource busy oleg229-server: detected 4 online CPUs by sysfs oleg229-server: Force libcfs to create 2 CPU partitions Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre Starting client: oleg229-client.virtnet: -o user_xattr,flock oleg229-server@tcp:/lustre /mnt/lustre2 PASS 109 (519s) == sanityn test 110: do not grant another lock on resend ========================================================== 15:09:42 (1772309382) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: /mnt/lustre/f110.sanityn Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-02-28 15:09:42.000000000 -0500 Modify: 2026-02-28 15:09:42.000000000 -0500 Change: 2026-02-28 15:09:42.000000000 -0500 Birth: 2026-02-28 15:09:42.000000000 -0500 fail_loc=0 fail_val=0 Connected clients: oleg229-client.virtnet oleg229-client.virtnet PASS 110 (35s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 15:10:17 (1772309417) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 15:10:18 (1772309418) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 15:10:19 (1772309419) lustre-OST0000-osc-ffff8d64073a2000 active. lustre-OST0001-osc-ffff8d64073a2000 active. lustre-OST0000-osc-ffff8d64073a2000 active. lustre-OST0001-osc-ffff8d64073a2000 active. MGC192.168.202.129@tcp active. PASS 113 (2s) cleanup: ====================================================== == sanityn test complete, duration 4627 sec ============== 15:10:22 (1772309422) Stopping clients: oleg229-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg229-client.virtnet /mnt/lustre2 opts: