-----============= acceptance-small: sanityn ============----- Tue Jan 7 09:49:28 EST 2025 excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 Started clients oleg321-client.virtnet: 192.168.203.121@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt) oleg321-client.virtnet: executing check_config_client /mnt/lustre oleg321-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg321-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800b58af000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b63d9800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b58af000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b63d9800.idle_timeout=debug disable quota as required oleg321-server: oleg321-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181 s, 57.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6848] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 09:49:37 (1736261377) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (2s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 09:49:40 (1736261380) -rw-r--r-- 1 root root 0 Jan 7 09:49 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (3s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 09:49:42 (1736261382) -rw-r--r-- 1 root root 0 Jan 7 09:49 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (2s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 09:49:44 (1736261384) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (2s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 09:49:46 (1736261386) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (1s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 09:49:47 (1736261387) total 1 -rwxrwxrwx 1 root root 0 Jan 7 09:49 f2a -rwxrwxrwx 1 root root 0 Jan 7 09:49 f2b total 1 -rwxrwxrwx 1 root root 0 Jan 7 09:49 f2a -rwxrwxrwx 1 root root 0 Jan 7 09:49 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00256238 s, 200 kB/s PASS 2e (1s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 09:49:48 (1736261388) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 09:49:49 (1736261389) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0747359 s, 28.1 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (2s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 09:49:51 (1736261391) PASS 3 (1s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 09:49:52 (1736261392) PASS 4 (2s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 09:49:54 (1736261394) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (2s) == sanityn test 6: remove of open file on other node ============================================================================ 09:49:56 (1736261396) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (2s) == sanityn test 7: remove of open directory on other node ======================================================================= 09:49:58 (1736261398) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (1s) == sanityn test 8: remove of open special file on other node ==================================================================== 09:49:59 (1736261399) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (1s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 09:50:00 (1736261400) PASS 9a (2s) == sanityn test 9b: append to striped sparse file ======== 09:50:02 (1736261402) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00980089 s, 107 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.00985611 s, 0.3 kB/s Data read (expecting 'foo'): foo PASS 9b (1s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 09:50:03 (1736261403) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00197906 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00151343 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00135599 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0014621 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00156688 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00154644 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00120453 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00170232 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00136596 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00137846 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00139531 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00133714 s, 0.7 kB/s PASS 10a (2s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 09:50:05 (1736261405) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00269288 s, 1.1 MB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00516268 s, 793 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000153942 s, 20.0 MB/s PASS 10b (1s) == sanityn test 11: execution of file opened for write should return error ============================================================== 09:50:06 (1736261406) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6848 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (2s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 09:50:08 (1736261408) warning: '-runtime' deprecated, use '-t runtime' instead - link 3553 (time 1736261418.50 total 10.00 last 355.27) - link 6746 (time 1736261428.50 total 20.00 last 319.26) - link 10000 (time 1736261438.06 total 29.56 last 340.38) - link 13428 (time 1736261448.07 total 39.56 last 342.76) - link 16916 (time 1736261458.07 total 49.56 last 348.78) - link 20000 (time 1736261467.51 total 59.01 last 326.52) - link 23235 (time 1736261477.51 total 69.01 last 323.42) - link 26257 (time 1736261487.51 total 79.01 last 302.19) - link 28367 (time 1736261497.52 total 89.02 last 210.84) - link 30000 (time 1736261504.99 total 96.49 last 218.56) total: 31138 link in 100.00 seconds: 311.38 ops/second using seed 3428909611 running for 100 seconds - stat 10000 (time 1736261411 ; total 2 ; last 2) - stat 20000 (time 1736261414 ; total 5 ; last 3) - stat 30000 (time 1736261416 ; total 7 ; last 2) - stat 40000 (time 1736261419 ; total 10 ; last 3) - stat 50000 (time 1736261421 ; total 12 ; last 2) - stat 60000 (time 1736261424 ; total 15 ; last 3) - stat 70000 (time 1736261426 ; total 17 ; last 2) - stat 80000 (time 1736261429 ; total 20 ; last 3) - stat 90000 (time 1736261431 ; total 22 ; last 2) - stat 100000 (time 1736261434 ; total 25 ; last 3) - stat 110000 (time 1736261436 ; total 27 ; last 2) - stat 120000 (time 1736261439 ; total 30 ; last 3) - stat 130000 (time 1736261441 ; total 32 ; last 2) - stat 140000 (time 1736261443 ; total 34 ; last 2) - stat 150000 (time 1736261446 ; total 37 ; last 3) - stat 160000 (time 1736261448 ; total 39 ; last 2) - stat 170000 (time 1736261451 ; total 42 ; last 3) - stat 180000 (time 1736261453 ; total 44 ; last 2) - stat 190000 (time 1736261456 ; total 47 ; last 3) - stat 200000 (time 1736261458 ; total 49 ; last 2) - stat 210000 (time 1736261463 ; total 54 ; last 5) - stat 220000 (time 1736261469 ; total 60 ; last 6) - stat 230000 (time 1736261474 ; total 65 ; last 5) - stat 240000 (time 1736261480 ; total 71 ; last 6) - stat 250000 (time 1736261487 ; total 78 ; last 7) - stat 260000 (time 1736261494 ; total 85 ; last 7) - stat 270000 (time 1736261503 ; total 94 ; last 9) total: 279924 stats in 100 seconds: 2799.239990 stats/second - unlinked 0 (time 1736261459 ; total 0 ; last 0) - unlinked 10000 (time 1736261510 ; total 51 ; last 51) - unlinked 20000 (time 1736261532 ; total 73 ; last 22) - unlinked 30000 (time 1736261554 ; total 95 ; last 22) unlink(/mnt/lustre2/lockdir/lockfile31138) error: No such file or directory total: 31137 unlinks in 97 seconds: 321.000000 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (20444) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (20447) - No such process PASS 12 (149s) == sanityn test 13: test directory page revocation ======= 09:52:37 (1736261557) f13 f13 PASS 13 (2s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 09:52:39 (1736261559) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6848 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4260: 21829 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (1s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 09:52:40 (1736261560) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4260: 22456 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (2s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 09:52:42 (1736261562) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 23082 Terminated $DIR1/$tdir/sleep 60 PASS 14b (1s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 09:52:43 (1736261563) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 23711 Terminated $DIR1/$tdir/sleep 60 PASS 14c (1s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 09:52:45 (1736261565) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 24339 Terminated $DIR1/$tdir/sleep 60 PASS 14d (1s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 09:52:46 (1736261566) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7520256 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7520256kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (11s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 09:52:57 (1736261577) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.098542 s, 21.3 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0676321 s, 31.0 MB/s Chance of close/open is 1 in 50 Seed set to 1578 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read 000004[1] 1736261578.439540 write 0x04b4e6 thru 0x058849 (0xd364 bytes) truncating to largest ever: 0x9ce2df 000005[0] 1736261578.447982 trunc from 0x05884a to 0x9ce2de (0x975a95 bytes) 000006[1] 1736261578.470643 read 0x34a761 thru 0x34fdc8 (0x5668 bytes) 000007[0] 1736261578.482597 trunc from 0x9ce2df to 0x86dbf9 (0x1606e5 bytes) 000008[1] 1736261578.498237 mapread 0x6a7008 thru 0x6aeba8 (0x7ba1 bytes) 000010[0] 1736261578.526043 read 0x13f6e3 thru 0x1459ef (0x630d bytes) 000011[0] 1736261578.530120 write 0x044570 thru 0x0503b3 (0xbe44 bytes) 000012[1] 1736261578.546152 read 0x0a99b4 thru 0x0ad89f (0x3eec bytes) 000013[0] 1736261578.560849 mapwrite 0x3c8175 thru 0x3ced0d (0x6b99 bytes) 000014[1] 1736261578.610746 read 0x5e8812 thru 0x5ef51c (0x6d0b bytes) 000019[0] 1736261578.635051 mapread 0x146e1f thru 0x14ba1c (0x4bfe bytes) 000020[1] 1736261578.643800 write 0x7bd3c2 thru 0x7cb203 (0xde42 bytes) 000021[0] 1736261578.654903 mapwrite 0x47bcd5 thru 0x48ab30 (0xee5c bytes) 000023[0] 1736261578.711521 read 0x19982a thru 0x19c658 (0x2e2f bytes) 000024[1] 1736261578.754777 write 0x14124e thru 0x1456c3 (0x4476 bytes) 000027[0] 1736261578.790955 trunc from 0x86dbfa to 0x950798 (0xe2b9f bytes) 000028[1] 1736261578.806263 write 0x8a81a5 thru 0x8b6b02 (0xe95e bytes) 000030[0] 1736261578.836999 mapread 0x768f56 thru 0x775948 (0xc9f3 bytes) 000032[1] 1736261578.882599 write 0x7ef8b2 thru 0x7f194c (0x209b bytes) 000033[1] 1736261578.892270 trunc from 0x950799 to 0x56d193 (0x3e3605 bytes) 000038[1] 1736261578.915479 trunc from 0x56d194 to 0x2826ba (0x2eaad9 bytes) 000040[0] 1736261578.935508 mapread 0x12b222 thru 0x1363f5 (0xb1d4 bytes) 000041[0] 1736261578.968244 read 0x0bb755 thru 0x0bc242 (0xaee bytes) 000042[0] 1736261578.986768 write 0x0ca545 thru 0x0cf32f (0x4deb bytes) 000043[1] 1736261579.002126 mapread 0x08c964 thru 0x093fdb (0x7678 bytes) 000045[1] 1736261579.029185 read 0x1c1122 thru 0x1caa77 (0x9956 bytes) 000046[1] 1736261579.034991 read 0x1689c7 thru 0x17728e (0xe8c8 bytes) truncating to largest ever: 0x9fe1ff 000047[0] 1736261579.041604 trunc from 0x2826bb to 0x9fe1fe (0x77bb44 bytes) 000048[0] 1736261579.061708 mapwrite 0x37e60f thru 0x38a036 (0xba28 bytes) 000049[1] 1736261579.131970 read 0x22015c thru 0x229ffd (0x9ea2 bytes) 000051[0] 1736261579.158163 read 0x437d4a thru 0x43bb69 (0x3e20 bytes) 000052[1] 1736261579.185749 trunc from 0x9fe1ff to 0x8e6556 (0x117ca8 bytes) 000053[0] 1736261579.209975 mapwrite 0x78b081 thru 0x78d8c5 (0x2845 bytes) 000054[0] 1736261579.243129 mapread 0x40bfd4 thru 0x41a779 (0xe7a6 bytes) 000055[1] 1736261579.280956 write 0x2a7631 thru 0x2b218b (0xab5b bytes) 000056[1] 1736261579.292750 mapread 0x115466 thru 0x1205f8 (0xb193 bytes) 000058[1] 1736261579.357859 mapread 0x7936dc thru 0x79a924 (0x7249 bytes) 000060[0] 1736261579.364651 write 0x10de9f thru 0x11484c (0x69ae bytes) 000063[1] 1736261579.397245 mapread 0x7330f6 thru 0x736908 (0x3813 bytes) 000064[0] 1736261579.420915 mapwrite 0x9114fb thru 0x9195d1 (0x80d7 bytes) 000065[1] 1736261579.473941 read 0x800a83 thru 0x80f3bc (0xe93a bytes) 000068[0] 1736261579.493472 mapread 0x87da97 thru 0x882235 (0x479f bytes) 000072[0] 1736261579.503454 mapread 0x181f37 thru 0x18e647 (0xc711 bytes) 000073[1] 1736261579.531880 write 0x66a5b8 thru 0x66a607 (0x50 bytes) 000074[1] 1736261579.541772 write 0x4280f9 thru 0x4363dd (0xe2e5 bytes) 000075[0] 1736261579.549516 read 0x161391 thru 0x1645ce (0x323e bytes) 000076[1] 1736261579.560538 trunc from 0x9195d2 to 0x2f6bfb (0x6229d6 bytes) 000077[1] 1736261579.578005 read 0x036661 thru 0x043fd0 (0xd970 bytes) 000078[0] 1736261579.599605 mapwrite 0x532215 thru 0x53dfab (0xbd97 bytes) 000079[0] 1736261579.658328 mapread 0x22f9bb thru 0x23f91a (0xff60 bytes) 000081[1] 1736261579.668724 mapread 0x191766 thru 0x1928b4 (0x114f bytes) 000083[0] 1736261579.685533 write 0x59c720 thru 0x5a1a41 (0x5322 bytes) 000084[1] 1736261579.688566 mapread 0x2f1c58 thru 0x2f21f7 (0x5a0 bytes) 000086[0] 1736261579.702025 write 0x4767f1 thru 0x4818eb (0xb0fb bytes) 000087[0] 1736261579.712190 mapwrite 0x55cbc3 thru 0x569c54 (0xd092 bytes) 000088[1] 1736261579.748031 mapwrite 0x461431 thru 0x4634d9 (0x20a9 bytes) 000092[0] 1736261579.799762 mapread 0x1dd6f0 thru 0x1eaf59 (0xd86a bytes) 000094[1] 1736261579.831538 write 0x6a8c5e thru 0x6ac769 (0x3b0c bytes) 000098[1] 1736261579.858934 read 0x0243e4 thru 0x02a834 (0x6451 bytes) 000099[0] 1736261579.877150 trunc from 0x6ac76a to 0x4efc8e (0x1bcadb bytes) 000100[0] 1736261579.905753 trunc from 0x4efc8f to 0x9a5447 (0x4b57b9 bytes) 000101[1] 1736261579.919519 mapwrite 0x76ecc5 thru 0x77180e (0x2b4a bytes) 000104[1] 1736261579.976897 write 0x1b41c2 thru 0x1b7570 (0x33af bytes) 000105[1] 1736261579.982750 write 0x4b3e48 thru 0x4c1b4d (0xdd06 bytes) 000107[0] 1736261580.027513 read 0x7a1b1b thru 0x7acfad (0xb493 bytes) 000108[1] 1736261580.047250 mapwrite 0x76275a thru 0x76e657 (0xbefe bytes) 000110[0] 1736261580.105185 mapread 0x44ed9e thru 0x455360 (0x65c3 bytes) 000111[1] 1736261580.151430 mapread 0x830935 thru 0x83f9bd (0xf089 bytes) 000114[0] 1736261580.187532 trunc from 0x9a5448 to 0x317c7e (0x68d7c9 bytes) 000116[0] 1736261580.200823 read 0x0bcba5 thru 0x0bdf93 (0x13ef bytes) 000118[1] 1736261580.230373 trunc from 0x317c7f to 0x0b3412 (0x26486c bytes) 000119[0] 1736261580.256898 trunc from 0x0b3413 to 0x652ba9 (0x59f797 bytes) 000122[0] 1736261580.279392 mapwrite 0x8e12d8 thru 0x8e2f4f (0x1c78 bytes) 000123[0] 1736261580.346137 read 0x7e349c thru 0x7eed10 (0xb875 bytes) 000124[1] 1736261580.348545 read 0x48ef0e thru 0x493f4e (0x5041 bytes) 000125[1] 1736261580.363170 mapwrite 0x62b68e thru 0x62bafd (0x470 bytes) 000127[0] 1736261580.398028 read 0x748dd5 thru 0x74bfce (0x31fa bytes) 000129[0] 1736261580.407005 mapread 0x0b3ce1 thru 0x0c0eec (0xd20c bytes) 000132[0] 1736261580.442717 mapwrite 0x40e5b5 thru 0x4182f3 (0x9d3f bytes) 000134[1] 1736261580.488183 trunc from 0x8e2f50 to 0x8fb312 (0x183c3 bytes) 000135[1] 1736261580.504988 write 0x49880c thru 0x4a2b60 (0xa355 bytes) 000138[1] 1736261580.531304 read 0x334958 thru 0x33cc67 (0x8310 bytes) 000139[1] 1736261580.547874 read 0x1ca074 thru 0x1d70bf (0xd04c bytes) 000140[1] 1736261580.564155 mapread 0x694a18 thru 0x6a37f9 (0xede2 bytes) 000141[0] 1736261580.569789 mapwrite 0x259f4b thru 0x25c4a7 (0x255d bytes) 000142[0] 1736261580.620987 read 0x5b8c13 thru 0x5c48f6 (0xbce4 bytes) 000144[1] 1736261580.648784 trunc from 0x8fb313 to 0x69b607 (0x25fd0b bytes) 000145[1] 1736261580.667704 write 0x92b1d2 thru 0x939110 (0xdf3f bytes) 000146[1] 1736261580.672915 mapwrite 0x7bb7a8 thru 0x7c6bba (0xb413 bytes) 000152[0] 1736261580.738528 mapread 0x038665 thru 0x045519 (0xceb5 bytes) 000153[1] 1736261580.770748 read 0x88e45c thru 0x898e7f (0xaa24 bytes) 000154[0] 1736261580.781253 write 0x0d108d thru 0x0de812 (0xd786 bytes) 000159[1] 1736261580.813514 mapwrite 0x8c2551 thru 0x8cfe01 (0xd8b1 bytes) 000160[1] 1736261580.883939 read 0x3abeec thru 0x3ad682 (0x1797 bytes) 000162[0] 1736261580.912935 mapwrite 0x12dac4 thru 0x13b4b3 (0xd9f0 bytes) 000165[0] 1736261580.971273 trunc from 0x939111 to 0x15ae2a (0x7de2e6 bytes) 000167[0] 1736261580.992942 mapread 0x055412 thru 0x05bf8f (0x6b7e bytes) 000169[0] 1736261581.008967 write 0x3029c7 thru 0x309077 (0x66b1 bytes) 000170[1] 1736261581.009923 mapread 0x1d3610 thru 0x1e03f6 (0xcde7 bytes) 000171[1] 1736261581.062442 mapwrite 0x9e6846 thru 0x9e9a2f (0x31ea bytes) 000172[0] 1736261581.109524 read 0x90b6c0 thru 0x915a82 (0xa3c3 bytes) 000175[1] 1736261581.132171 write 0x2053ad thru 0x20d714 (0x8368 bytes) 000178[1] 1736261581.136993 mapread 0x0d86a1 thru 0x0d9de6 (0x1746 bytes) 000179[1] 1736261581.144010 write 0x4a760a thru 0x4b70d1 (0xfac8 bytes) 000180[0] 1736261581.155126 mapread 0x713f22 thru 0x71e31a (0xa3f9 bytes) 000183[0] 1736261581.202459 write 0x09cffe thru 0x0aad0f (0xdd12 bytes) 000184[1] 1736261581.221442 trunc from 0x9e9a30 to 0x423066 (0x5c69c9 bytes) 000185[0] 1736261581.238618 read 0x08a8c5 thru 0x08fb26 (0x5262 bytes) 000186[0] 1736261581.251888 read 0x226621 thru 0x227846 (0x1226 bytes) 000187[1] 1736261581.267394 write 0x7ee4cd thru 0x7f392c (0x5460 bytes) 000188[1] 1736261581.273724 mapread 0x4df658 thru 0x4e4ff8 (0x59a1 bytes) 000189[0] 1736261581.290589 mapwrite 0x360833 thru 0x3679b4 (0x7182 bytes) 000191[0] 1736261581.359463 mapwrite 0x3a3a63 thru 0x3a9f11 (0x64af bytes) 000193[0] 1736261581.384831 mapwrite 0x58cfe9 thru 0x5936c2 (0x66da bytes) 000197[0] 1736261581.416671 read 0x6b6aeb thru 0x6beffe (0x8514 bytes) 000198[1] 1736261581.425141 trunc from 0x7f392d to 0x39a065 (0x4598c7 bytes) 000199[0] 1736261581.450259 read 0x137e6e thru 0x13ba5d (0x3bf0 bytes) 000200[1] 1736261581.468101 mapwrite 0x03ae38 thru 0x03c000 (0x11c9 bytes) 000201[1] 1736261581.511982 mapwrite 0x15d47b thru 0x16c1c8 (0xed4e bytes) 000203[1] 1736261581.572952 trunc from 0x39a066 to 0x86e06f (0x4d400a bytes) 000204[0] 1736261581.581079 write 0x16d386 thru 0x17b3fe (0xe079 bytes) 000206[1] 1736261581.609133 read 0x5d63f4 thru 0x5d76d8 (0x12e5 bytes) 000208[0] 1736261581.643839 trunc from 0x86e070 to 0x977136 (0x1090c7 bytes) 000209[1] 1736261581.664505 write 0x8635bf thru 0x86d4cb (0x9f0d bytes) 000211[0] 1736261581.688883 write 0x83b4b3 thru 0x83c8ae (0x13fc bytes) 000212[0] 1736261581.738086 mapwrite 0x165197 thru 0x170787 (0xb5f1 bytes) 000217[0] 1736261581.814600 write 0x9bd935 thru 0x9caed0 (0xd59c bytes) 000219[0] 1736261581.824192 mapwrite 0x6dd369 thru 0x6e5315 (0x7fad bytes) 000220[0] 1736261581.855820 mapwrite 0x2f58a1 thru 0x2ff1f1 (0x9951 bytes) 000221[1] 1736261581.887000 trunc from 0x9caed1 to 0x8b0972 (0x11a55e bytes) 000222[0] 1736261581.915598 mapwrite 0x6a9997 thru 0x6b96cc (0xfd36 bytes) 000223[1] 1736261581.986043 write 0x271ccc thru 0x27f81a (0xdb4f bytes) 000225[1] 1736261581.999111 mapread 0x1ebe39 thru 0x1edade (0x1ca6 bytes) 000227[1] 1736261582.003419 read 0x0ce2b7 thru 0x0ce6eb (0x435 bytes) 000232[0] 1736261582.063131 mapread 0x2fb496 thru 0x30479b (0x9306 bytes) 000233[0] 1736261582.100126 write 0x3ebdc9 thru 0x3f3eea (0x8122 bytes) 000234[1] 1736261582.112079 write 0x6c205e thru 0x6c69e7 (0x498a bytes) 000235[1] 1736261582.123661 write 0x71636f thru 0x71d73e (0x73d0 bytes) 000236[0] 1736261582.137961 mapread 0x024948 thru 0x029da0 (0x5459 bytes) 000237[1] 1736261582.163482 read 0x59b9c0 thru 0x59d46c (0x1aad bytes) 000238[0] 1736261582.171898 mapwrite 0x87a49a thru 0x888c4f (0xe7b6 bytes) 000239[0] 1736261582.243715 trunc from 0x8b0973 to 0x1df6d7 (0x6d129b bytes) 000240[0] 1736261582.261877 read 0x00ae61 thru 0x018f12 (0xe0b2 bytes) 000241[1] 1736261582.284573 mapread 0x1a19c1 thru 0x1ac3d8 (0xaa18 bytes) 000243[1] 1736261582.315749 mapread 0x1af779 thru 0x1b9e69 (0xa6f1 bytes) 000245[0] 1736261582.342939 mapread 0x12a600 thru 0x1300c3 (0x5ac4 bytes) 000248[1] 1736261582.362834 read 0x120d07 thru 0x124d0f (0x4009 bytes) 000249[0] 1736261582.370399 mapwrite 0x06c6e3 thru 0x07928d (0xcbab bytes) 000251[1] 1736261582.393679 write 0x40f8d8 thru 0x41306f (0x3798 bytes) 000252[0] 1736261582.398597 write 0x415639 thru 0x423a9d (0xe465 bytes) 000254[0] 1736261582.414957 read 0x0553ee thru 0x05d6f5 (0x8308 bytes) 000259[1] 1736261582.441044 write 0x9cd66b thru 0x9d4815 (0x71ab bytes) 000260[0] 1736261582.452852 trunc from 0x9d4816 to 0x60f37f (0x3c5496 bytes) 000262[0] 1736261582.479114 read 0x0e200f thru 0x0ea2b3 (0x82a5 bytes) 000263[0] 1736261582.484970 write 0x4bb392 thru 0x4c7afc (0xc76b bytes) 000264[0] 1736261582.497769 trunc from 0x60f380 to 0x771179 (0x161dfa bytes) 000265[1] 1736261582.512592 write 0x40a97d thru 0x40ca2a (0x20ae bytes) 000266[1] 1736261582.537218 trunc from 0x77117a to 0x04ab71 (0x726608 bytes) 000269[0] 1736261582.567456 mapread 0x039691 thru 0x043595 (0x9f05 bytes) 000270[1] 1736261582.597403 mapread 0x00c736 thru 0x0196de (0xcfa9 bytes) 000272[1] 1736261582.625502 write 0x0031eb thru 0x004ec4 (0x1cda bytes) 000274[0] 1736261582.639864 mapread 0x00a102 thru 0x0175d6 (0xd4d5 bytes) 000276[1] 1736261582.686750 mapread 0x0307d1 thru 0x038e8f (0x86bf bytes) 000278[0] 1736261582.711999 mapread 0x013711 thru 0x020589 (0xce79 bytes) 000282[0] 1736261582.758185 read 0x01d094 thru 0x01ed99 (0x1d06 bytes) 000283[0] 1736261582.758548 trunc from 0x04ab72 to 0x1064b3 (0xbb942 bytes) 000284[0] 1736261582.778362 read 0x0a8215 thru 0x0ac109 (0x3ef5 bytes) 000285[0] 1736261582.785387 mapread 0x0e27a6 thru 0x0f2486 (0xfce1 bytes) 000286[0] 1736261582.793264 mapwrite 0x5aac2f thru 0x5b28be (0x7c90 bytes) 000287[0] 1736261582.853363 mapwrite 0x538577 thru 0x54748e (0xef18 bytes) 000288[0] 1736261582.916869 mapread 0x0b3216 thru 0x0bca10 (0x97fb bytes) 000289[1] 1736261582.923637 trunc from 0x5b28bf to 0x40486c (0x1ae052 bytes) 000290[0] 1736261582.945031 read 0x0f297d thru 0x0fce17 (0xa49b bytes) 000296[1] 1736261582.969560 read 0x0902c9 thru 0x09671e (0x6456 bytes) 000299[1] 1736261583.002183 trunc from 0x40486d to 0x0890fd (0x37b76f bytes) 000303[1] 1736261583.033322 mapwrite 0x62f1b6 thru 0x63675f (0x75aa bytes) 000304[0] 1736261583.095687 mapwrite 0x0fd6b4 thru 0x107bcd (0xa51a bytes) 000305[0] 1736261583.176209 trunc from 0x636760 to 0x6cbdc5 (0x95666 bytes) 000306[1] 1736261583.185728 mapread 0x6b5ae9 thru 0x6be036 (0x854e bytes) 000308[1] 1736261583.222918 trunc from 0x6cbdc6 to 0x2f073a (0x3db68b bytes) 000312[0] 1736261583.261138 mapread 0x2782db thru 0x27fe55 (0x7b7b bytes) 000315[1] 1736261583.296169 mapwrite 0x5ffcd3 thru 0x60acd0 (0xaffe bytes) 000316[1] 1736261583.360335 trunc from 0x60acd1 to 0x950c53 (0x345f83 bytes) 000317[1] 1736261583.375998 trunc from 0x950c54 to 0x2308e0 (0x720373 bytes) 000318[1] 1736261583.392762 write 0x7bd727 thru 0x7c7861 (0xa13b bytes) 000320[0] 1736261583.407194 trunc from 0x7c7862 to 0x7c56e9 (0x2178 bytes) 000321[0] 1736261583.428874 mapread 0x66d85b thru 0x673674 (0x5e1a bytes) 000322[1] 1736261583.447370 trunc from 0x7c56ea to 0x951378 (0x18bc8f bytes) 000323[1] 1736261583.461017 mapread 0x853197 thru 0x85f75e (0xc5c8 bytes) 000326[1] 1736261583.492249 mapread 0x01c9dd thru 0x021b3c (0x5160 bytes) 000327[0] 1736261583.505150 trunc from 0x951379 to 0x22d7e3 (0x723b95 bytes) 000328[1] 1736261583.520636 mapwrite 0x2407e0 thru 0x2481ed (0x7a0e bytes) 000330[0] 1736261583.569392 mapread 0x048179 thru 0x053755 (0xb5dd bytes) 000331[1] 1736261583.594767 mapwrite 0x67ac17 thru 0x67de19 (0x3203 bytes) 000333[1] 1736261583.639215 mapwrite 0x36daa2 thru 0x37b35b (0xd8ba bytes) 000334[0] 1736261583.684622 read 0x0bdb7b thru 0x0bfa5b (0x1ee1 bytes) 000336[1] 1736261583.698008 read 0x0d93e0 thru 0x0df670 (0x6291 bytes) 000337[1] 1736261583.716478 trunc from 0x67de1a to 0x1bc382 (0x4c1a97 bytes) 000339[1] 1736261583.740100 read 0x070811 thru 0x073677 (0x2e67 bytes) 000341[1] 1736261583.761753 write 0x72f0f3 thru 0x7363dd (0x72eb bytes) 000342[0] 1736261583.767982 trunc from 0x7363de to 0x6ec99a (0x49a43 bytes) 000343[1] 1736261583.785991 trunc from 0x6ec99b to 0x45f433 (0x28d567 bytes) 000345[1] 1736261583.804885 mapwrite 0x3e4da3 thru 0x3ebd4f (0x6fad bytes) 000348[0] 1736261583.848396 mapwrite 0x43b9a8 thru 0x43eb79 (0x31d2 bytes) 000350[0] 1736261583.899615 mapwrite 0x435d10 thru 0x43fa7e (0x9d6f bytes) 000352[0] 1736261583.948839 mapwrite 0x7fb5e3 thru 0x80295d (0x737b bytes) 000353[0] 1736261584.017230 mapwrite 0x35f7e6 thru 0x36a906 (0xb121 bytes) 000354[1] 1736261584.037316 read 0x375f07 thru 0x379e04 (0x3efe bytes) 000355[0] 1736261584.051152 trunc from 0x80295e to 0x045d89 (0x7bcbd4 bytes) 000356[1] 1736261584.062242 mapwrite 0x851df4 thru 0x856c38 (0x4e45 bytes) 000358[0] 1736261584.114520 write 0x781526 thru 0x78fab5 (0xe590 bytes) 000361[0] 1736261584.136293 trunc from 0x856c39 to 0x0ca635 (0x78c603 bytes) 000362[0] 1736261584.153015 trunc from 0x0ca636 to 0x4b0718 (0x3e60e3 bytes) 000364[0] 1736261584.168752 read 0x0b6086 thru 0x0be0bd (0x8038 bytes) 000366[1] 1736261584.193449 read 0x470d8f thru 0x471179 (0x3eb bytes) 000369[0] 1736261584.208326 mapwrite 0x84a597 thru 0x84b090 (0xafa bytes) 000370[0] 1736261584.252760 mapread 0x2afab0 thru 0x2b4c1f (0x5170 bytes) 000371[1] 1736261584.266480 mapwrite 0x6ed1fd thru 0x6fad86 (0xdb8a bytes) 000372[0] 1736261584.317685 read 0x4122ad thru 0x421699 (0xf3ed bytes) 000374[0] 1736261584.339834 mapwrite 0x9522f5 thru 0x95e758 (0xc464 bytes) 000375[1] 1736261584.377985 mapread 0x6452c5 thru 0x651405 (0xc141 bytes) 000379[1] 1736261584.422423 mapwrite 0x6ad374 thru 0x6b6857 (0x94e4 bytes) 000381[1] 1736261584.474427 read 0x5e7f8b thru 0x5f67e9 (0xe85f bytes) 000383[1] 1736261584.505388 mapwrite 0x84ee69 thru 0x85bdec (0xcf84 bytes) 000384[0] 1736261584.528575 read 0x289799 thru 0x28cd04 (0x356c bytes) 000385[1] 1736261584.557411 mapwrite 0x4b6fca thru 0x4bd140 (0x6177 bytes) 000386[0] 1736261584.599286 read 0x4b1a89 thru 0x4ba201 (0x8779 bytes) 000388[1] 1736261584.616822 trunc from 0x95e759 to 0x6499a7 (0x314db1 bytes) 000389[1] 1736261584.634677 write 0x1a7293 thru 0x1b5656 (0xe3c4 bytes) 000394[0] 1736261584.650746 read 0x41f45a thru 0x42782f (0x83d6 bytes) 000398[1] 1736261584.667638 write 0x2c1a44 thru 0x2c7127 (0x56e4 bytes) 000404[0] 1736261584.699204 mapwrite 0x5608cc thru 0x56cf86 (0xc6bb bytes) 000405[0] 1736261584.741343 mapwrite 0x245962 thru 0x24ca99 (0x7138 bytes) 000412[0] 1736261584.799199 mapwrite 0x6198be thru 0x625120 (0xb863 bytes) 000413[0] 1736261584.838162 read 0x1a8771 thru 0x1a897a (0x20a bytes) 000414[1] 1736261584.855281 read 0x25ddca thru 0x26a41d (0xc654 bytes) 000418[1] 1736261584.871466 mapwrite 0x4e33aa thru 0x4ed323 (0x9f7a bytes) 000419[1] 1736261584.929732 mapread 0x5d4168 thru 0x5e393b (0xf7d4 bytes) 000422[1] 1736261584.961164 mapwrite 0x52b8a4 thru 0x52d5c3 (0x1d20 bytes) 000426[0] 1736261585.031301 write 0x8bc785 thru 0x8cb8db (0xf157 bytes) 000427[0] 1736261585.042743 write 0x8de819 thru 0x8e9406 (0xabee bytes) 000429[0] 1736261585.058531 write 0x1594e9 thru 0x15fade (0x65f6 bytes) 000430[1] 1736261585.080948 mapwrite 0x419065 thru 0x41cc82 (0x3c1e bytes) 000435[1] 1736261585.144944 mapread 0x498215 thru 0x4a2280 (0xa06c bytes) 000436[1] 1736261585.165557 write 0x1467ba thru 0x152bd1 (0xc418 bytes) 000437[1] 1736261585.176534 mapread 0x0fb3a8 thru 0x1089bf (0xd618 bytes) 000439[1] 1736261585.198053 trunc from 0x8e9407 to 0x4d32d1 (0x416135 bytes) 000440[0] 1736261585.211539 write 0x9e1bdc thru 0x9ec25f (0xa684 bytes) 000444[0] 1736261585.247939 read 0x51ca54 thru 0x52b584 (0xeb31 bytes) 000445[1] 1736261585.260848 write 0x8d2c77 thru 0x8e2236 (0xf5c0 bytes) 000446[0] 1736261585.268785 trunc from 0x9ec260 to 0x9e56de (0x6b81 bytes) 000447[0] 1736261585.289846 read 0x429a66 thru 0x42bcf0 (0x228b bytes) 000451[1] 1736261585.336242 write 0x3b328c thru 0x3b51d6 (0x1f4b bytes) 000453[0] 1736261585.367584 mapwrite 0x222566 thru 0x22cb01 (0xa59c bytes) 000454[0] 1736261585.425549 write 0x60b793 thru 0x617f75 (0xc7e3 bytes) 000455[1] 1736261585.431429 write 0x8ffc41 thru 0x90accc (0xb08c bytes) 000456[1] 1736261585.452354 write 0x79a367 thru 0x7a3af1 (0x978b bytes) 000457[0] 1736261585.462898 trunc from 0x9e56df to 0x6878ff (0x35dddf bytes) 000458[1] 1736261585.485227 read 0x50b224 thru 0x5142bb (0x9098 bytes) 000459[0] 1736261585.498655 read 0x44996e thru 0x450a3d (0x70d0 bytes) 000460[1] 1736261585.519364 read 0x32cc32 thru 0x3300eb (0x34ba bytes) 000464[1] 1736261585.537537 mapwrite 0x2cc54a thru 0x2d9538 (0xcfef bytes) 000467[0] 1736261585.609266 write 0x001eb3 thru 0x011355 (0xf4a3 bytes) 000471[1] 1736261585.630259 write 0x63a64d thru 0x63ae5e (0x812 bytes) 000472[1] 1736261585.650501 read 0x5681d9 thru 0x56b88c (0x36b4 bytes) 000474[1] 1736261585.651110 read 0x3ffaeb thru 0x406f38 (0x744e bytes) 000476[1] 1736261585.675057 mapwrite 0x27beb8 thru 0x284a76 (0x8bbf bytes) 000478[1] 1736261585.717686 mapwrite 0x7f040c thru 0x7fc5b7 (0xc1ac bytes) 000479[1] 1736261585.781021 mapread 0x6407f4 thru 0x648517 (0x7d24 bytes) 000482[0] 1736261585.792975 write 0x308bb3 thru 0x3180cd (0xf51b bytes) 000483[1] 1736261585.804899 mapwrite 0x928192 thru 0x931b77 (0x99e6 bytes) 000484[1] 1736261585.866835 write 0x64b370 thru 0x650be6 (0x5877 bytes) 000486[1] 1736261585.873241 trunc from 0x931b78 to 0x079741 (0x8b8436 bytes) 000487[1] 1736261585.887136 read 0x01168c thru 0x015e40 (0x47b5 bytes) 000488[1] 1736261585.897000 mapwrite 0x2ebd6e thru 0x2f5571 (0x9804 bytes) 000490[0] 1736261585.939587 mapwrite 0x914341 thru 0x91c2c1 (0x7f81 bytes) 000492[0] 1736261586.002912 trunc from 0x91c2c2 to 0x5b1fcf (0x36a2f2 bytes) 000493[1] 1736261586.015427 trunc from 0x5b1fd0 to 0x51c829 (0x957a6 bytes) 000494[0] 1736261586.027939 trunc from 0x51c82a to 0x1c09fc (0x35be2d bytes) 000495[0] 1736261586.041456 read 0x191a3e thru 0x198069 (0x662c bytes) 000496[0] 1736261586.050721 mapwrite 0x2a1032 thru 0x2a6da6 (0x5d75 bytes) 000498[0] 1736261586.086942 write 0x0467aa thru 0x04f53f (0x8d96 bytes) 000500[0] 1736261586.102150 mapwrite 0x08de17 thru 0x0978b2 (0x9a9c bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 1586 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[1] 1736261586.365102 write_OD 0x9ea000 thru 0x9f6fff (0xd000 bytes) 000002[1] 1736261586.394589 read_OD 0x7e7000 thru 0x7f2fff (0xc000 bytes) truncating to largest ever: 0x47617b 000004[0] 1736261586.399816 trunc from 0x9f7000 to 0x47617a (0x580e85 bytes) 000005[1] 1736261586.407651 mapread 0x107000 thru 0x107318 (0x319 bytes) 000009[0] 1736261586.416182 read 0x09a000 thru 0x0a505d (0xb05e bytes) 000010[0] 1736261586.418935 mapwrite 0x314000 thru 0x315fd0 (0x1fd1 bytes) 000011[0] 1736261586.450506 mapwrite 0x7eb000 thru 0x7ef685 (0x4686 bytes) 000012[0] 1736261586.482891 write 0x072000 thru 0x07925a (0x725b bytes) 000013[0] 1736261586.491789 mapwrite 0x151000 thru 0x160689 (0xf68a bytes) 000014[1] 1736261586.525653 write_OD 0x205000 thru 0x211fff (0xd000 bytes) 000017[1] 1736261586.550252 mapread 0x6d4000 thru 0x6da08b (0x608c bytes) 000018[0] 1736261586.556534 write 0x007000 thru 0x014110 (0xd111 bytes) 000019[1] 1736261586.564104 mapread 0x7a0000 thru 0x7a1e76 (0x1e77 bytes) 000024[0] 1736261586.575977 write 0x854000 thru 0x85d545 (0x9546 bytes) 000026[0] 1736261586.580486 read 0x1b6000 thru 0x1c43b4 (0xe3b5 bytes) 000028[0] 1736261586.613903 trunc from 0x85d546 to 0x211180 (0x64c3c5 bytes) truncating to largest ever: 0x52104e 000030[0] 1736261586.634162 trunc from 0x211181 to 0x52104d (0x30fecd bytes) truncating to largest ever: 0x66c2d5 000031[0] 1736261586.642740 trunc from 0x52104e to 0x66c2d4 (0x14b287 bytes) 000033[0] 1736261586.647335 read 0x1d2000 thru 0x1d35da (0x15db bytes) 000034[0] 1736261586.647897 mapread 0x59c000 thru 0x5a4b9a (0x8b9b bytes) 000036[0] 1736261586.663742 mapwrite 0x885000 thru 0x8903a8 (0xb3a9 bytes) 000039[0] 1736261586.710942 mapread 0x40f000 thru 0x418d8b (0x9d8c bytes) 000047[0] 1736261586.721516 write 0x3a0000 thru 0x3a5347 (0x5348 bytes) 000048[1] 1736261586.722111 write_OD 0x157000 thru 0x158fff (0x2000 bytes) 000050[0] 1736261586.742457 read 0x47b000 thru 0x48a05b (0xf05c bytes) truncating to largest ever: 0x89b662 000051[0] 1736261586.762893 trunc from 0x8903a9 to 0x89b661 (0xb2b9 bytes) 000052[1] 1736261586.781161 read_OD 0x868000 thru 0x86cfff (0x5000 bytes) 000054[1] 1736261586.794814 mapwrite 0x131000 thru 0x135346 (0x4347 bytes) 000055[0] 1736261586.856526 mapread 0x686000 thru 0x688e5a (0x2e5b bytes) 000056[0] 1736261586.868949 mapwrite 0x532000 thru 0x53232b (0x32c bytes) 000057[1] 1736261586.892518 mapwrite 0x870000 thru 0x87e199 (0xe19a bytes) truncating to largest ever: 0x90e8f6 000058[0] 1736261586.935043 trunc from 0x89b662 to 0x90e8f5 (0x73294 bytes) 000059[0] 1736261586.946800 write 0x24b000 thru 0x25967b (0xe67c bytes) 000060[0] 1736261586.953523 trunc from 0x90e8f6 to 0x656b9c (0x2b7d59 bytes) 000061[0] 1736261586.959014 read 0x54b000 thru 0x54ec76 (0x3c77 bytes) 000062[1] 1736261586.969056 mapwrite 0x196000 thru 0x19daba (0x7abb bytes) 000063[1] 1736261587.002240 write_OD 0x623000 thru 0x62cfff (0xa000 bytes) 000064[0] 1736261587.017386 mapread 0x506000 thru 0x51250c (0xc50d bytes) 000065[1] 1736261587.035429 read_OD 0x00d000 thru 0x00efff (0x2000 bytes) 000066[0] 1736261587.037202 mapread 0x152000 thru 0x15c8be (0xa8bf bytes) 000067[1] 1736261587.051922 write_OD 0x612000 thru 0x61cfff (0xb000 bytes) 000068[1] 1736261587.076928 mapread 0x441000 thru 0x443eca (0x2ecb bytes) 000069[1] 1736261587.108549 trunc from 0x656b9d to 0x595ae9 (0xc10b3 bytes) 000071[0] 1736261587.136684 mapread 0x2da000 thru 0x2e447e (0xa47f bytes) 000072[0] 1736261587.173212 read 0x2f2000 thru 0x301b90 (0xfb91 bytes) 000075[0] 1736261587.190468 mapwrite 0x852000 thru 0x85cc06 (0xac07 bytes) 000079[1] 1736261587.248494 write_OD 0x9d5000 thru 0x9d8fff (0x4000 bytes) 000080[0] 1736261587.268393 write 0x285000 thru 0x2915ee (0xc5ef bytes) 000081[0] 1736261587.275992 read 0x17e000 thru 0x17e0c2 (0xc3 bytes) 000085[1] 1736261587.308489 read_OD 0x2d0000 thru 0x2dcfff (0xd000 bytes) 000086[1] 1736261587.326744 mapwrite 0x3a0000 thru 0x3a5928 (0x5929 bytes) 000088[1] 1736261587.396569 trunc from 0x9d9000 to 0x0d0a6d (0x908592 bytes) 000089[1] 1736261587.414579 mapwrite 0x224000 thru 0x230ca5 (0xcca6 bytes) 000090[0] 1736261587.471040 mapread 0x058000 thru 0x064284 (0xc285 bytes) 000094[0] 1736261587.501573 read 0x12d000 thru 0x13877c (0xb77d bytes) 000098[0] 1736261587.511296 read 0x185000 thru 0x18943a (0x443b bytes) 000101[1] 1736261587.511704 write_OD 0x3a1000 thru 0x3a9fff (0x9000 bytes) 000102[0] 1736261587.534293 mapread 0x30c000 thru 0x311e89 (0x5e8a bytes) 000103[0] 1736261587.546818 mapread 0x388000 thru 0x38ac2b (0x2c2c bytes) 000104[0] 1736261587.553268 read 0x1a6000 thru 0x1b33b5 (0xd3b6 bytes) 000105[1] 1736261587.566530 mapwrite 0x25c000 thru 0x26a4b8 (0xe4b9 bytes) 000108[1] 1736261587.621775 trunc from 0x3aa000 to 0x6049be (0x25a9bf bytes) 000109[1] 1736261587.633235 write_OD 0x884000 thru 0x888fff (0x5000 bytes) 000113[1] 1736261587.655937 write_OD 0x669000 thru 0x672fff (0xa000 bytes) 000114[1] 1736261587.682588 read_OD 0x6b9000 thru 0x6b9fff (0x1000 bytes) 000117[1] 1736261587.688173 mapwrite 0x89b000 thru 0x8a1780 (0x6781 bytes) 000119[1] 1736261587.742979 read_OD 0x54a000 thru 0x54afff (0x1000 bytes) 000120[1] 1736261587.751502 mapwrite 0x1a0000 thru 0x1aeb6b (0xeb6c bytes) 000121[0] 1736261587.833812 read 0x655000 thru 0x6638a0 (0xe8a1 bytes) 000123[0] 1736261587.858274 mapwrite 0x423000 thru 0x42ad31 (0x7d32 bytes) 000125[1] 1736261587.917434 write_OD 0x0ce000 thru 0x0d7fff (0xa000 bytes) 000126[1] 1736261587.942853 read_OD 0x389000 thru 0x38bfff (0x3000 bytes) 000128[0] 1736261587.946119 read 0x663000 thru 0x66b458 (0x8459 bytes) 000130[1] 1736261587.963566 mapwrite 0x483000 thru 0x48ff1c (0xcf1d bytes) 000131[0] 1736261588.022009 write 0x517000 thru 0x525217 (0xe218 bytes) 000133[1] 1736261588.038338 read_OD 0x533000 thru 0x537fff (0x5000 bytes) 000134[1] 1736261588.049559 read_OD 0x65f000 thru 0x665fff (0x7000 bytes) 000137[0] 1736261588.054510 trunc from 0x8a1781 to 0x5f0f48 (0x2b0838 bytes) 000138[1] 1736261588.071412 trunc from 0x5f0f49 to 0x65736a (0x66422 bytes) 000139[1] 1736261588.080756 mapread 0x0f6000 thru 0x0f946d (0x346e bytes) 000140[0] 1736261588.101998 mapread 0x1ed000 thru 0x1f96cb (0xc6cc bytes) 000141[1] 1736261588.140285 mapwrite 0x995000 thru 0x9a4536 (0xf537 bytes) 000142[0] 1736261588.213457 read 0x258000 thru 0x265450 (0xd451 bytes) 000143[0] 1736261588.263993 mapread 0x280000 thru 0x283766 (0x3767 bytes) 000145[1] 1736261588.264548 mapread 0x462000 thru 0x468e57 (0x6e58 bytes) 000147[0] 1736261588.281223 read 0x554000 thru 0x55eb5a (0xab5b bytes) 000148[1] 1736261588.281520 mapwrite 0x740000 thru 0x745196 (0x5197 bytes) 000149[0] 1736261588.325540 mapwrite 0x257000 thru 0x25a36e (0x336f bytes) 000151[1] 1736261588.379656 write_OD 0x1e3000 thru 0x1e6fff (0x4000 bytes) 000152[0] 1736261588.401895 write 0x528000 thru 0x5284f2 (0x4f3 bytes) 000154[0] 1736261588.420029 write 0x7c9000 thru 0x7d1bb2 (0x8bb3 bytes) 000155[1] 1736261588.426513 mapwrite 0x84b000 thru 0x854913 (0x9914 bytes) 000156[1] 1736261588.494348 write_OD 0x015000 thru 0x01afff (0x6000 bytes) 000157[1] 1736261588.524435 mapread 0x58b000 thru 0x5917dd (0x67de bytes) 000158[1] 1736261588.541423 trunc from 0x9a4537 to 0x16beb3 (0x838683 bytes) 000159[1] 1736261588.553480 trunc from 0x16beb4 to 0x8b7347 (0x74b494 bytes) 000164[1] 1736261588.583718 trunc from 0x8b7348 to 0x582930 (0x334a17 bytes) 000165[0] 1736261588.589685 write 0x4d7000 thru 0x4debe6 (0x7be7 bytes) truncating to largest ever: 0x96165d 000167[0] 1736261588.604283 trunc from 0x582931 to 0x96165c (0x3ded2c bytes) 000168[1] 1736261588.611803 read_OD 0x22b000 thru 0x231fff (0x7000 bytes) 000171[1] 1736261588.625525 write_OD 0x883000 thru 0x883fff (0x1000 bytes) 000172[0] 1736261588.651513 read 0x6e6000 thru 0x6e96da (0x36db bytes) 000173[0] 1736261588.666144 read 0x78d000 thru 0x7912c0 (0x42c1 bytes) 000174[1] 1736261588.666419 mapwrite 0x35d000 thru 0x36b35c (0xe35d bytes) 000176[0] 1736261588.726651 trunc from 0x96165d to 0x4b2760 (0x4aeefc bytes) 000177[0] 1736261588.744203 mapwrite 0x0ff000 thru 0x108d78 (0x9d79 bytes) 000178[1] 1736261588.784397 write_OD 0x53b000 thru 0x547fff (0xd000 bytes) 000180[1] 1736261588.807372 mapwrite 0x8c8000 thru 0x8cfb36 (0x7b37 bytes) 000183[0] 1736261588.855978 mapread 0x207000 thru 0x214b8f (0xdb90 bytes) 000184[1] 1736261588.884371 write_OD 0x083000 thru 0x083fff (0x1000 bytes) 000188[0] 1736261588.906764 mapread 0x759000 thru 0x76463a (0xb63b bytes) 000189[1] 1736261588.927895 write_OD 0x523000 thru 0x52dfff (0xb000 bytes) 000193[1] 1736261588.945523 mapread 0x1b2000 thru 0x1b9046 (0x7047 bytes) 000195[1] 1736261588.957106 trunc from 0x8cfb37 to 0x758942 (0x1771f4 bytes) 000196[1] 1736261588.969546 mapwrite 0x2b0000 thru 0x2b7264 (0x7265 bytes) 000199[1] 1736261589.004473 write_OD 0x251000 thru 0x251fff (0x1000 bytes) 000201[0] 1736261589.016910 mapread 0x681000 thru 0x68fa36 (0xea37 bytes) 000202[0] 1736261589.035601 write 0x1bc000 thru 0x1c4a5d (0x8a5e bytes) 000203[1] 1736261589.046756 mapwrite 0x4be000 thru 0x4c0181 (0x2182 bytes) 000205[1] 1736261589.077836 read_OD 0x430000 thru 0x434fff (0x5000 bytes) 000207[0] 1736261589.085071 read 0x74b000 thru 0x74da97 (0x2a98 bytes) 000209[1] 1736261589.089055 read_OD 0x644000 thru 0x651fff (0xe000 bytes) 000210[0] 1736261589.091648 mapwrite 0x945000 thru 0x952045 (0xd046 bytes) 000211[0] 1736261589.136132 trunc from 0x952046 to 0x639d1e (0x318327 bytes) 000212[1] 1736261589.144499 mapread 0x39c000 thru 0x3a38cd (0x78ce bytes) truncating to largest ever: 0x9c7dad 000215[1] 1736261589.161442 trunc from 0x639d1f to 0x9c7dac (0x38e08e bytes) 000216[0] 1736261589.173825 mapwrite 0x95e000 thru 0x96630b (0x830c bytes) 000217[1] 1736261589.214982 mapread 0x925000 thru 0x933e80 (0xee81 bytes) 000218[1] 1736261589.239448 mapwrite 0x8bb000 thru 0x8bdc34 (0x2c35 bytes) 000220[1] 1736261589.281737 read_OD 0x8b9000 thru 0x8bcfff (0x4000 bytes) 000221[1] 1736261589.285110 trunc from 0x9c7dad to 0x2156d3 (0x7b26d9 bytes) 000223[0] 1736261589.294139 trunc from 0x2156d4 to 0x0b011f (0x1655b4 bytes) 000224[0] 1736261589.301644 mapread 0x09c000 thru 0x0a2a39 (0x6a3a bytes) 000225[0] 1736261589.311137 mapwrite 0x3c9000 thru 0x3ccfcb (0x3fcc bytes) 000226[1] 1736261589.343608 read_OD 0x1f4000 thru 0x1f8fff (0x5000 bytes) 000228[0] 1736261589.353990 mapread 0x125000 thru 0x12fdb3 (0xadb4 bytes) 000230[0] 1736261589.372606 write 0x8e1000 thru 0x8e4f72 (0x3f73 bytes) 000231[1] 1736261589.383951 trunc from 0x8e4f73 to 0x901d03 (0x1cd91 bytes) 000232[1] 1736261589.392619 write_OD 0x559000 thru 0x564fff (0xc000 bytes) 000233[0] 1736261589.414973 read 0x8da000 thru 0x8e7139 (0xd13a bytes) 000234[0] 1736261589.419276 trunc from 0x901d04 to 0x1c9a31 (0x7382d2 bytes) 000235[1] 1736261589.433675 read_OD 0x1ae000 thru 0x1b8fff (0xb000 bytes) 000240[0] 1736261589.442535 read 0x02f000 thru 0x03408d (0x508e bytes) 000241[1] 1736261589.452965 read_OD 0x183000 thru 0x188fff (0x6000 bytes) 000242[0] 1736261589.455251 write 0x4b6000 thru 0x4c0797 (0xa798 bytes) 000246[1] 1736261589.467665 read_OD 0x1ff000 thru 0x203fff (0x5000 bytes) 000247[1] 1736261589.476987 write_OD 0x60f000 thru 0x61bfff (0xd000 bytes) 000248[1] 1736261589.497154 trunc from 0x61c000 to 0x49d165 (0x17ee9a bytes) 000250[1] 1736261589.504426 mapread 0x413000 thru 0x41321f (0x220 bytes) 000251[0] 1736261589.515886 write 0x87d000 thru 0x87ee84 (0x1e85 bytes) 000252[0] 1736261589.523389 read 0x443000 thru 0x44eb4d (0xbb4e bytes) 000260[1] 1736261589.541756 trunc from 0x87ee85 to 0x5cfffd (0x2aee87 bytes) 000263[0] 1736261589.555121 write 0x5dc000 thru 0x5e2d7e (0x6d7f bytes) 000266[0] 1736261589.563997 trunc from 0x5e2d7f to 0x16e70b (0x474673 bytes) 000267[1] 1736261589.568887 read_OD 0x004000 thru 0x012fff (0xf000 bytes) 000268[1] 1736261589.576816 read_OD 0x041000 thru 0x04afff (0xa000 bytes) 000270[1] 1736261589.580969 mapwrite 0x216000 thru 0x220982 (0xa983 bytes) 000271[1] 1736261589.627343 trunc from 0x220983 to 0x21c8ce (0x40b4 bytes) 000272[0] 1736261589.636076 mapwrite 0x70e000 thru 0x715ebc (0x7ebd bytes) 000273[0] 1736261589.680588 mapwrite 0x2cd000 thru 0x2da466 (0xd467 bytes) 000274[1] 1736261589.727406 trunc from 0x715ebd to 0x7e106e (0xcb1b2 bytes) 000279[0] 1736261589.754294 trunc from 0x7e106f to 0x97ea4b (0x19d9dd bytes) 000280[1] 1736261589.769418 read_OD 0x690000 thru 0x69dfff (0xe000 bytes) 000283[1] 1736261589.778293 read_OD 0x453000 thru 0x457fff (0x5000 bytes) 000284[0] 1736261589.781651 write 0x207000 thru 0x20fa20 (0x8a21 bytes) 000287[1] 1736261589.806730 mapread 0x4dd000 thru 0x4eb0fd (0xe0fe bytes) 000289[1] 1736261589.833547 write_OD 0x15a000 thru 0x167fff (0xe000 bytes) 000292[0] 1736261589.865454 trunc from 0x97ea4c to 0x3d7465 (0x5a75e6 bytes) 000293[1] 1736261589.873221 mapwrite 0x0fb000 thru 0x106616 (0xb617 bytes) 000295[1] 1736261589.923604 mapwrite 0x74f000 thru 0x753582 (0x4583 bytes) 000296[1] 1736261589.957789 mapread 0x746000 thru 0x74b903 (0x5904 bytes) 000299[0] 1736261589.969981 mapread 0x597000 thru 0x597c4e (0xc4f bytes) 000302[0] 1736261589.978876 mapwrite 0x3b3000 thru 0x3ba30e (0x730f bytes) 000303[0] 1736261590.009908 mapwrite 0x054000 thru 0x055ec6 (0x1ec7 bytes) 000306[0] 1736261590.051507 read_OD 0x335000 thru 0x336fff (0x2000 bytes) 000312[1] 1736261590.058244 write_OD 0x475000 thru 0x47cfff (0x8000 bytes) 000314[0] 1736261590.078985 trunc from 0x753583 to 0x693c88 (0xbf8fa bytes) 000315[1] 1736261590.092748 trunc from 0x693c89 to 0x8497b1 (0x1b5b29 bytes) 000317[1] 1736261590.105001 mapread 0x15d000 thru 0x169ab3 (0xcab4 bytes) 000318[1] 1736261590.124278 mapread 0x3b9000 thru 0x3bee72 (0x5e73 bytes) 000320[0] 1736261590.132940 trunc from 0x8497b2 to 0x2536ea (0x5f60c7 bytes) 000322[0] 1736261590.141515 mapread 0x08a000 thru 0x096cc2 (0xccc3 bytes) 000325[1] 1736261590.161122 read_OD 0x1dc000 thru 0x1e3fff (0x8000 bytes) 000326[1] 1736261590.171238 write_OD 0x96b000 thru 0x976fff (0xc000 bytes) 000332[0] 1736261590.201908 mapread 0x162000 thru 0x167a36 (0x5a37 bytes) 000334[1] 1736261590.205339 write_OD 0x667000 thru 0x66cfff (0x6000 bytes) 000335[0] 1736261590.223729 trunc from 0x977000 to 0x2a6e96 (0x6d0169 bytes) 000337[1] 1736261590.234107 trunc from 0x2a6e97 to 0x9a396b (0x6fcad5 bytes) 000340[1] 1736261590.253879 mapwrite 0x7fd000 thru 0x8064a7 (0x94a8 bytes) 000341[1] 1736261590.326208 mapwrite 0x40a000 thru 0x40f4ab (0x54ac bytes) 000343[0] 1736261590.355632 read_OD 0x497000 thru 0x49efff (0x8000 bytes) 000345[0] 1736261590.373465 mapread 0x03d000 thru 0x03e2ed (0x12ee bytes) 000346[0] 1736261590.400623 mapwrite 0x5ce000 thru 0x5d87f5 (0xa7f6 bytes) 000347[1] 1736261590.466833 trunc from 0x9a396c to 0x41f4b4 (0x5844b7 bytes) 000352[0] 1736261590.494397 trunc from 0x41f4b5 to 0x6416c7 (0x222213 bytes) 000356[1] 1736261590.525851 mapwrite 0x149000 thru 0x154af4 (0xbaf5 bytes) 000359[1] 1736261590.577794 mapread 0x4ac000 thru 0x4af237 (0x3238 bytes) 000360[0] 1736261590.589996 mapread 0x21e000 thru 0x22544b (0x744c bytes) 000362[1] 1736261590.608562 read_OD 0x1a9000 thru 0x1a9fff (0x1000 bytes) 000364[0] 1736261590.613063 read_OD 0x580000 thru 0x585fff (0x6000 bytes) 000365[1] 1736261590.616371 read_OD 0x4d5000 thru 0x4d6fff (0x2000 bytes) 000366[0] 1736261590.619252 mapwrite 0x7e0000 thru 0x7e43a7 (0x43a8 bytes) 000369[1] 1736261590.675545 mapread 0x6c1000 thru 0x6cdcdb (0xccdc bytes) 000371[0] 1736261590.701681 write_OD 0x29e000 thru 0x2a4fff (0x7000 bytes) 000372[1] 1736261590.727761 write_OD 0x3fd000 thru 0x408fff (0xc000 bytes) 000373[0] 1736261590.761254 mapwrite 0x278000 thru 0x27e81e (0x681f bytes) 000376[1] 1736261590.790907 write_OD 0x36a000 thru 0x370fff (0x7000 bytes) 000377[1] 1736261590.815944 trunc from 0x7e43a8 to 0x649a08 (0x19a99f bytes) 000380[1] 1736261590.828771 trunc from 0x649a09 to 0x3fbf49 (0x24dabf bytes) 000382[1] 1736261590.843155 mapread 0x17d000 thru 0x187ff7 (0xaff8 bytes) 000383[1] 1736261590.862591 mapread 0x2a0000 thru 0x2a9bc5 (0x9bc6 bytes) 000384[0] 1736261590.886235 mapwrite 0x90c000 thru 0x90f2e4 (0x32e5 bytes) 000386[0] 1736261590.930720 mapwrite 0x7ac000 thru 0x7aeadc (0x2add bytes) 000389[0] 1736261590.985166 mapread 0x76c000 thru 0x7782d3 (0xc2d4 bytes) 000391[1] 1736261591.019211 mapwrite 0x687000 thru 0x689d62 (0x2d63 bytes) 000396[0] 1736261591.064429 read_OD 0x1cf000 thru 0x1d6fff (0x8000 bytes) 000399[1] 1736261591.078249 read_OD 0x2a7000 thru 0x2aefff (0x8000 bytes) 000400[0] 1736261591.082409 mapwrite 0x854000 thru 0x85f26a (0xb26b bytes) 000406[0] 1736261591.161786 mapread 0x51d000 thru 0x52c8d0 (0xf8d1 bytes) 000407[1] 1736261591.170719 read_OD 0x3ea000 thru 0x3f0fff (0x7000 bytes) 000412[1] 1736261591.188893 trunc from 0x90f2e5 to 0x5b01ae (0x35f136 bytes) 000414[1] 1736261591.210682 trunc from 0x5b01af to 0x5a2689 (0xdb25 bytes) 000415[1] 1736261591.216102 write_OD 0x301000 thru 0x306fff (0x6000 bytes) 000417[0] 1736261591.249811 trunc from 0x5a268a to 0x734f58 (0x1928cf bytes) 000420[1] 1736261591.271326 read_OD 0x4de000 thru 0x4e7fff (0xa000 bytes) 000421[1] 1736261591.279495 mapwrite 0x320000 thru 0x324209 (0x420a bytes) 000422[0] 1736261591.324277 write_OD 0x5ab000 thru 0x5b8fff (0xe000 bytes) 000424[0] 1736261591.351455 mapwrite 0x6b2000 thru 0x6b64b3 (0x44b4 bytes) 000426[0] 1736261591.398993 trunc from 0x734f59 to 0x2412dd (0x4f3c7b bytes) 000429[1] 1736261591.414907 mapread 0x105000 thru 0x111c41 (0xcc42 bytes) 000430[1] 1736261591.443643 write_OD 0x48c000 thru 0x48dfff (0x2000 bytes) 000431[1] 1736261591.466953 write_OD 0x252000 thru 0x25efff (0xd000 bytes) 000432[1] 1736261591.486358 write_OD 0x9c3000 thru 0x9c3fff (0x1000 bytes) 000434[0] 1736261591.506798 write_OD 0x2d7000 thru 0x2ddfff (0x7000 bytes) 000435[1] 1736261591.526034 trunc from 0x9c4000 to 0x7bb960 (0x20869f bytes) 000436[0] 1736261591.537559 mapwrite 0x61e000 thru 0x62ab26 (0xcb27 bytes) 000439[1] 1736261591.585034 mapread 0x248000 thru 0x250d4b (0x8d4c bytes) 000440[1] 1736261591.611501 read_OD 0x124000 thru 0x129fff (0x6000 bytes) 000444[1] 1736261591.615625 mapread 0x478000 thru 0x48785c (0xf85d bytes) 000445[1] 1736261591.639202 write_OD 0x045000 thru 0x050fff (0xc000 bytes) 000446[1] 1736261591.671055 mapwrite 0x2c2000 thru 0x2cbd15 (0x9d16 bytes) 000453[0] 1736261591.728287 trunc from 0x7bb961 to 0x200ad7 (0x5bae89 bytes) 000454[0] 1736261591.744708 mapwrite 0x671000 thru 0x67eccc (0xdccd bytes) 000455[0] 1736261591.815534 mapwrite 0x294000 thru 0x298b67 (0x4b68 bytes) 000456[1] 1736261591.849734 write_OD 0x285000 thru 0x285fff (0x1000 bytes) 000457[0] 1736261591.872833 trunc from 0x67eccd to 0x9245cc (0x2a5900 bytes) 000459[1] 1736261591.886802 read_OD 0x222000 thru 0x22dfff (0xc000 bytes) 000460[1] 1736261591.894853 mapwrite 0x3c0000 thru 0x3c44ca (0x44cb bytes) 000461[1] 1736261591.945227 trunc from 0x9245cd to 0x46c27d (0x4b834f bytes) 000462[0] 1736261591.958673 mapwrite 0x21d000 thru 0x2295d6 (0xc5d7 bytes) 000463[0] 1736261592.012936 mapread 0x18b000 thru 0x18cee6 (0x1ee7 bytes) 000464[0] 1736261592.023553 read_OD 0x19e000 thru 0x1acfff (0xf000 bytes) 000465[0] 1736261592.030538 mapread 0x055000 thru 0x0558c3 (0x8c4 bytes) 000466[1] 1736261592.044825 mapread 0x1ed000 thru 0x1fb94b (0xe94c bytes) 000467[0] 1736261592.068906 mapread 0x3c0000 thru 0x3c1712 (0x1713 bytes) 000468[1] 1736261592.069225 mapread 0x0de000 thru 0x0de823 (0x824 bytes) 000470[1] 1736261592.071149 write_OD 0x80a000 thru 0x815fff (0xc000 bytes) 000472[0] 1736261592.099137 trunc from 0x816000 to 0x864490 (0x4e491 bytes) 000474[0] 1736261592.114819 mapwrite 0x744000 thru 0x74f65e (0xb65f bytes) 000475[1] 1736261592.158932 trunc from 0x864491 to 0x19223a (0x6d2256 bytes) 000476[1] 1736261592.167390 mapread 0x0a3000 thru 0x0b26b2 (0xf6b3 bytes) 000479[0] 1736261592.197049 trunc from 0x19223b to 0x51e0a9 (0x38be6f bytes) 000481[1] 1736261592.213109 read_OD 0x332000 thru 0x332fff (0x1000 bytes) 000483[1] 1736261592.220976 write_OD 0x113000 thru 0x11efff (0xc000 bytes) 000484[1] 1736261592.240278 trunc from 0x51e0aa to 0x0e32ef (0x43adba bytes) 000485[1] 1736261592.248746 write_OD 0x82d000 thru 0x837fff (0xb000 bytes) 000489[0] 1736261592.273625 read_OD 0x237000 thru 0x244fff (0xe000 bytes) 000490[0] 1736261592.276026 trunc from 0x838000 to 0x1601cd (0x6d7e32 bytes) 000492[1] 1736261592.292213 read_OD 0x02e000 thru 0x02ffff (0x2000 bytes) 000493[1] 1736261592.298570 trunc from 0x1601ce to 0x6c31be (0x562ff1 bytes) 000497[0] 1736261592.321178 write_OD 0x80e000 thru 0x816fff (0x9000 bytes) 000499[1] 1736261592.337058 read_OD 0x45e000 thru 0x464fff (0x7000 bytes) 000500[1] 1736261592.341591 mapwrite 0x9aa000 thru 0x9aaf5d (0xf5e bytes) All operations completed A-OK! PASS 16a (16s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 09:53:13 (1736261593) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.100511 s, 20.9 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0820713 s, 25.6 MB/s Chance of close/open is 1 in 50 Seed set to 1594 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read 000002[1] 1736261594.570875 mapwrite 0x561bca thru 0x562036 (0x46d bytes) 000006[1] 1736261594.629848 read 0x080ab8 thru 0x081164 (0x6ad bytes) 000007[0] 1736261594.632543 mapread 0x09ae66 thru 0x09c594 (0x172f bytes) 000008[0] 1736261594.652335 mapwrite 0x1c5ca7 thru 0x1c60a0 (0x3fa bytes) truncating to largest ever: 0x834900 000010[1] 1736261594.685676 trunc from 0x562037 to 0x8348ff (0x2d28c9 bytes) 000013[0] 1736261594.711128 read 0x6e7c4d thru 0x6e880c (0xbc0 bytes) 000018[1] 1736261594.731650 mapread 0x4e49f5 thru 0x4e5951 (0xf5d bytes) 000020[0] 1736261594.745709 trunc from 0x834900 to 0x02e86c (0x806093 bytes) 000022[1] 1736261594.765345 mapread 0x01522d thru 0x015a8c (0x860 bytes) truncating to largest ever: 0x96b56d 000025[0] 1736261594.787108 trunc from 0x02e86d to 0x96b56c (0x93cd00 bytes) 000027[0] 1736261594.808727 mapwrite 0x44b9d5 thru 0x44d947 (0x1f73 bytes) 000030[1] 1736261594.853353 read 0x1b7999 thru 0x1b7bbc (0x224 bytes) 000034[0] 1736261594.882599 mapwrite 0x3129c1 thru 0x314490 (0x1ad0 bytes) truncating to largest ever: 0x9b24e7 000035[1] 1736261594.913819 trunc from 0x96b56d to 0x9b24e6 (0x46f7a bytes) 000038[1] 1736261594.935125 mapwrite 0x5dd13e thru 0x5def77 (0x1e3a bytes) 000039[0] 1736261594.998523 mapwrite 0x2c5ab0 thru 0x2c6dce (0x131f bytes) 000042[1] 1736261595.057211 read 0x131e75 thru 0x132d34 (0xec0 bytes) 000043[1] 1736261595.071810 mapwrite 0x76139b thru 0x7626a6 (0x130c bytes) 000044[0] 1736261595.098054 mapwrite 0x84889d thru 0x849d60 (0x14c4 bytes) 000045[1] 1736261595.124499 mapwrite 0x7e8eab thru 0x7e9158 (0x2ae bytes) 000046[1] 1736261595.144530 trunc from 0x9b24e7 to 0x31abdf (0x697907 bytes) 000047[0] 1736261595.160448 write 0x1c42c4 thru 0x1c4d77 (0xab4 bytes) 000048[1] 1736261595.172592 mapread 0x011fe3 thru 0x013f49 (0x1f67 bytes) 000052[1] 1736261595.195133 read 0x301b09 thru 0x3031b3 (0x16ab bytes) 000054[1] 1736261595.205044 read 0x220c3c thru 0x22131b (0x6e0 bytes) 000055[1] 1736261595.213733 mapwrite 0x08c669 thru 0x08e13d (0x1ad5 bytes) 000056[1] 1736261595.249736 read 0x2af745 thru 0x2b0f91 (0x184d bytes) 000057[1] 1736261595.255775 mapwrite 0x48af25 thru 0x48cb94 (0x1c70 bytes) 000058[1] 1736261595.302149 trunc from 0x48cb95 to 0x96fc07 (0x4e3073 bytes) 000061[0] 1736261595.314875 mapwrite 0x8530f0 thru 0x854ae3 (0x19f4 bytes) 000062[1] 1736261595.359442 mapwrite 0x431691 thru 0x431ccd (0x63d bytes) 000063[0] 1736261595.397346 read 0x5e5271 thru 0x5e600a (0xd9a bytes) 000064[1] 1736261595.406197 trunc from 0x96fc08 to 0x0acf48 (0x8c2cbf bytes) 000070[0] 1736261595.431553 mapwrite 0x82bf71 thru 0x82cafd (0xb8d bytes) 000072[0] 1736261595.486678 trunc from 0x82cafe to 0x8f5921 (0xc8e24 bytes) 000073[0] 1736261595.493376 mapwrite 0x37a6e8 thru 0x37a9ad (0x2c6 bytes) 000074[1] 1736261595.520111 write 0x33c630 thru 0x33d4d0 (0xea1 bytes) 000076[1] 1736261595.536449 read 0x2dd036 thru 0x2ddff9 (0xfc4 bytes) 000077[0] 1736261595.567866 mapread 0x6a3a77 thru 0x6a432c (0x8b6 bytes) 000078[0] 1736261595.576968 read 0x3b803a thru 0x3b810b (0xd2 bytes) 000080[0] 1736261595.610004 mapread 0x45b4e8 thru 0x45b679 (0x192 bytes) 000081[0] 1736261595.610383 write 0x1ebd6f thru 0x1ed132 (0x13c4 bytes) 000082[1] 1736261595.623645 mapread 0x472b1d thru 0x472dfe (0x2e2 bytes) 000085[0] 1736261595.626397 write 0x1eee36 thru 0x1efd4d (0xf18 bytes) 000086[0] 1736261595.630426 mapread 0x25e60a thru 0x25f0a7 (0xa9e bytes) 000087[1] 1736261595.634750 read 0x0eed28 thru 0x0efcf7 (0xfd0 bytes) 000089[0] 1736261595.643414 write 0x405c39 thru 0x406939 (0xd01 bytes) 000090[0] 1736261595.650348 read 0x2ceca5 thru 0x2cf67b (0x9d7 bytes) 000091[1] 1736261595.674981 mapwrite 0x1fc7bf thru 0x1fd355 (0xb97 bytes) 000093[1] 1736261595.712566 write 0x1053e4 thru 0x106be6 (0x1803 bytes) 000094[0] 1736261595.717339 trunc from 0x8f5922 to 0x34ee25 (0x5a6afc bytes) 000095[0] 1736261595.730643 mapwrite 0x39a830 thru 0x39bd23 (0x14f4 bytes) 000097[1] 1736261595.764820 trunc from 0x39bd24 to 0x60ee0a (0x2730e7 bytes) 000098[1] 1736261595.774566 read 0x2788de thru 0x27a854 (0x1f77 bytes) 000099[1] 1736261595.796635 write 0x9dc486 thru 0x9ddde5 (0x1960 bytes) 000101[1] 1736261595.802397 mapread 0x043461 thru 0x0449a0 (0x1540 bytes) 000102[1] 1736261595.818828 trunc from 0x9ddde6 to 0x8b72ef (0x126af6 bytes) 000104[0] 1736261595.831213 trunc from 0x8b72f0 to 0x8b60a9 (0x1246 bytes) 000105[0] 1736261595.853563 read 0x60dec5 thru 0x60fb4e (0x1c8a bytes) 000106[1] 1736261595.869967 trunc from 0x8b60aa to 0x08cf09 (0x8291a0 bytes) 000110[1] 1736261595.891919 read 0x02649d thru 0x026d86 (0x8ea bytes) 000111[0] 1736261595.897537 read 0x006ec2 thru 0x006f76 (0xb5 bytes) 000114[1] 1736261595.915274 mapwrite 0x98b6a9 thru 0x98ca5d (0x13b5 bytes) 000115[0] 1736261595.976591 trunc from 0x98ca5e to 0x81198f (0x17b0ce bytes) 000117[1] 1736261596.001526 mapwrite 0x023d77 thru 0x0246e4 (0x96e bytes) 000118[0] 1736261596.034916 read 0x70dbd2 thru 0x70dd13 (0x142 bytes) 000119[1] 1736261596.049814 mapread 0x63c083 thru 0x63ccad (0xc2b bytes) 000120[0] 1736261596.058063 mapwrite 0x710f37 thru 0x712345 (0x140f bytes) 000122[0] 1736261596.083508 trunc from 0x811990 to 0x142986 (0x6cf009 bytes) 000123[1] 1736261596.091549 mapread 0x06a571 thru 0x06b1a3 (0xc33 bytes) 000125[0] 1736261596.105621 read 0x08b3fd thru 0x08be7d (0xa81 bytes) 000128[0] 1736261596.115259 mapwrite 0x1409d9 thru 0x141fdd (0x1605 bytes) 000129[0] 1736261596.136636 trunc from 0x142987 to 0x7bf4a5 (0x67cb1f bytes) 000130[0] 1736261596.147013 mapread 0x225086 thru 0x225f67 (0xee2 bytes) 000132[1] 1736261596.160299 read 0x5b4584 thru 0x5b56bb (0x1138 bytes) 000133[0] 1736261596.168647 trunc from 0x7bf4a6 to 0x6be86e (0x100c37 bytes) 000134[0] 1736261596.181424 read 0x372ce7 thru 0x373417 (0x731 bytes) 000135[1] 1736261596.201044 write 0x7cdd9d thru 0x7cde3a (0x9e bytes) 000137[1] 1736261596.212867 read 0x35bbc2 thru 0x35d6a9 (0x1ae8 bytes) 000138[1] 1736261596.224647 write 0x6ae5dc thru 0x6b02f8 (0x1d1d bytes) 000139[0] 1736261596.244996 trunc from 0x7cde3b to 0x7fdf3f (0x30105 bytes) 000141[1] 1736261596.267721 mapread 0x4a5773 thru 0x4a70c2 (0x1950 bytes) 000144[1] 1736261596.281440 write 0x5efbb3 thru 0x5f04eb (0x939 bytes) 000145[0] 1736261596.292544 write 0x5aa033 thru 0x5abfc3 (0x1f91 bytes) 000147[0] 1736261596.308024 read 0x628c45 thru 0x6293f3 (0x7af bytes) 000149[0] 1736261596.318131 mapread 0x4a60bf thru 0x4a7aee (0x1a30 bytes) 000150[1] 1736261596.321591 read 0x67da36 thru 0x67f311 (0x18dc bytes) 000153[1] 1736261596.331609 trunc from 0x7fdf40 to 0x4ccbaf (0x331390 bytes) 000156[1] 1736261596.358139 trunc from 0x4ccbb0 to 0x42c558 (0xa0657 bytes) 000157[0] 1736261596.369827 trunc from 0x42c559 to 0x26c884 (0x1bfcd4 bytes) 000158[1] 1736261596.389065 read 0x24a709 thru 0x24b61f (0xf17 bytes) 000159[0] 1736261596.401047 trunc from 0x26c885 to 0x285e69 (0x195e5 bytes) 000160[0] 1736261596.413238 write 0x4b509c thru 0x4b6e9f (0x1e04 bytes) 000162[1] 1736261596.415475 read 0x0bb986 thru 0x0bbfa0 (0x61b bytes) 000164[0] 1736261596.424753 mapwrite 0x5762ea thru 0x577a7d (0x1794 bytes) 000169[0] 1736261596.459796 mapwrite 0x3bec0a thru 0x3c0120 (0x1517 bytes) 000170[1] 1736261596.481934 mapwrite 0x97e618 thru 0x980581 (0x1f6a bytes) 000171[1] 1736261596.540267 mapread 0x4536a9 thru 0x453fcb (0x923 bytes) 000172[0] 1736261596.540599 write 0x130554 thru 0x13218f (0x1c3c bytes) 000173[0] 1736261596.554225 mapread 0x24efbe thru 0x250bd8 (0x1c1b bytes) 000174[1] 1736261596.574905 write 0x4e08c1 thru 0x4e13ea (0xb2a bytes) 000175[0] 1736261596.584936 write 0x72d48a thru 0x72e0c2 (0xc39 bytes) 000184[1] 1736261596.626032 mapread 0x4beb08 thru 0x4c09c2 (0x1ebb bytes) 000186[0] 1736261596.639979 mapwrite 0x13445a thru 0x13614d (0x1cf4 bytes) 000187[1] 1736261596.663860 read 0x7f9ae6 thru 0x7fa5bf (0xada bytes) 000188[1] 1736261596.683336 trunc from 0x980582 to 0x7ec7a5 (0x193ddc bytes) 000191[1] 1736261596.704505 mapwrite 0x75cd00 thru 0x75d933 (0xc34 bytes) 000192[1] 1736261596.730938 write 0x2718f8 thru 0x27378d (0x1e96 bytes) truncating to largest ever: 0x9cb253 000194[0] 1736261596.737430 trunc from 0x7ec7a6 to 0x9cb252 (0x1deaad bytes) 000198[0] 1736261596.769280 mapread 0x9874d9 thru 0x987da7 (0x8cf bytes) 000199[0] 1736261596.779117 write 0x1ba3b1 thru 0x1baf2e (0xb7e bytes) 000200[0] 1736261596.780816 trunc from 0x9cb253 to 0x50b466 (0x4bfdec bytes) 000201[0] 1736261596.791792 mapwrite 0x0821bb thru 0x083c48 (0x1a8e bytes) 000202[1] 1736261596.809150 read 0x38e551 thru 0x38e9ce (0x47e bytes) 000203[0] 1736261596.824420 trunc from 0x50b467 to 0x6163a2 (0x10af3c bytes) 000204[1] 1736261596.839046 read 0x0d3dc4 thru 0x0d5273 (0x14b0 bytes) 000206[1] 1736261596.849567 mapwrite 0x846fab thru 0x847826 (0x87c bytes) 000209[1] 1736261596.920113 mapwrite 0x35f171 thru 0x36115c (0x1fec bytes) 000210[1] 1736261596.940834 trunc from 0x847827 to 0x8b5420 (0x6dbfa bytes) 000211[1] 1736261596.948285 mapwrite 0x446931 thru 0x447cfd (0x13cd bytes) 000212[0] 1736261596.972043 mapread 0x018ad2 thru 0x01a5e3 (0x1b12 bytes) 000213[1] 1736261596.986880 write 0x83b1c6 thru 0x83cb23 (0x195e bytes) 000214[1] 1736261596.997168 read 0x3fc48f thru 0x3fdd6f (0x18e1 bytes) 000215[1] 1736261596.997514 read 0x46d687 thru 0x46ef92 (0x190c bytes) 000216[0] 1736261597.010952 read 0x009f53 thru 0x00a8d5 (0x983 bytes) 000218[1] 1736261597.035075 trunc from 0x8b5421 to 0x358984 (0x55ca9c bytes) 000220[0] 1736261597.049315 mapread 0x18368f thru 0x18478d (0x10ff bytes) 000225[0] 1736261597.072402 mapwrite 0x8e8ada thru 0x8e9485 (0x9ac bytes) 000227[1] 1736261597.112158 trunc from 0x8e9486 to 0x78c91d (0x15cb68 bytes) 000228[1] 1736261597.123890 mapwrite 0x601fec thru 0x6032ac (0x12c1 bytes) 000229[1] 1736261597.144598 write 0x3d3bba thru 0x3d4c9f (0x10e6 bytes) 000230[0] 1736261597.152235 mapwrite 0x8e0e64 thru 0x8e2812 (0x19af bytes) 000232[0] 1736261597.193570 read 0x82ea67 thru 0x82ef29 (0x4c3 bytes) 000239[1] 1736261597.211486 read 0x8b917e thru 0x8ba8fc (0x177f bytes) 000241[1] 1736261597.225595 trunc from 0x8e2813 to 0x4d39bb (0x40ee57 bytes) 000242[0] 1736261597.241036 write 0x3ff743 thru 0x40114a (0x1a08 bytes) 000244[0] 1736261597.257256 trunc from 0x4d39bc to 0x58dc5e (0xba2a3 bytes) 000248[1] 1736261597.270101 mapwrite 0x3a4cca thru 0x3a62a1 (0x15d8 bytes) 000250[1] 1736261597.304227 mapread 0x4f0bae thru 0x4f24d7 (0x192a bytes) 000251[1] 1736261597.316446 read 0x25bb7c thru 0x25c295 (0x71a bytes) 000252[0] 1736261597.329718 write 0x236bed thru 0x238938 (0x1d4c bytes) 000254[1] 1736261597.342653 read 0x36ff0a thru 0x371318 (0x140f bytes) 000255[0] 1736261597.346367 trunc from 0x58dc5f to 0x6a5271 (0x117613 bytes) 000256[1] 1736261597.355131 read 0x322732 thru 0x32350f (0xdde bytes) 000257[1] 1736261597.365945 mapwrite 0x190c2d thru 0x192aa9 (0x1e7d bytes) 000258[0] 1736261597.395816 mapwrite 0x9d2451 thru 0x9d43fe (0x1fae bytes) 000259[0] 1736261597.436063 trunc from 0x9d43ff to 0x39b5ef (0x638e0f bytes) 000261[1] 1736261597.445943 trunc from 0x39b5f0 to 0x820698 (0x4850a9 bytes) 000263[0] 1736261597.473226 mapread 0x51707a thru 0x51743e (0x3c5 bytes) 000264[1] 1736261597.484922 write 0x2316d3 thru 0x232dce (0x16fc bytes) 000265[1] 1736261597.511197 write 0x8d77e4 thru 0x8d95f2 (0x1e0f bytes) 000266[0] 1736261597.516639 write 0x131892 thru 0x131ef4 (0x663 bytes) 000267[1] 1736261597.529140 read 0x3393c9 thru 0x33a660 (0x1298 bytes) 000269[0] 1736261597.555794 mapwrite 0x905b18 thru 0x9072ea (0x17d3 bytes) 000271[0] 1736261597.600007 trunc from 0x9072eb to 0x3f2e3a (0x5144b0 bytes) 000273[1] 1736261597.611984 mapwrite 0x47c4bd thru 0x47d881 (0x13c5 bytes) 000275[1] 1736261597.652714 trunc from 0x47d882 to 0x36c7ac (0x1110d5 bytes) 000279[1] 1736261597.668293 mapread 0x2c13fe thru 0x2c1997 (0x59a bytes) 000280[0] 1736261597.670021 trunc from 0x36c7ad to 0x5aa8c0 (0x23e114 bytes) 000281[1] 1736261597.684709 write 0x80a301 thru 0x80ad80 (0xa80 bytes) 000283[1] 1736261597.699140 read 0x292016 thru 0x292034 (0x1f bytes) 000284[1] 1736261597.714086 mapread 0x33d8c8 thru 0x33f754 (0x1e8d bytes) 000285[0] 1736261597.747194 mapwrite 0x344783 thru 0x3452ec (0xb6a bytes) 000289[0] 1736261597.793129 mapwrite 0x179056 thru 0x17945a (0x405 bytes) 000290[1] 1736261597.819849 read 0x5bad43 thru 0x5bc87c (0x1b3a bytes) 000291[0] 1736261597.829909 write 0x3e1f19 thru 0x3e260d (0x6f5 bytes) 000293[0] 1736261597.849816 read 0x7cb4b5 thru 0x7ccca3 (0x17ef bytes) 000295[0] 1736261597.866835 read 0x75dad4 thru 0x75e3c2 (0x8ef bytes) 000296[1] 1736261597.879364 mapwrite 0x9497ef thru 0x94b577 (0x1d89 bytes) 000297[0] 1736261597.911249 mapread 0x2fc9af thru 0x2fcebe (0x510 bytes) 000298[1] 1736261597.922372 read 0x353420 thru 0x3546ed (0x12ce bytes) 000300[0] 1736261597.957177 mapwrite 0x815570 thru 0x815fad (0xa3e bytes) 000303[1] 1736261597.997650 trunc from 0x94b578 to 0x5db440 (0x370137 bytes) 000305[0] 1736261598.011441 read 0x53a0ad thru 0x53a32c (0x280 bytes) 000306[0] 1736261598.026730 trunc from 0x5db441 to 0x6a87e2 (0xcd3a2 bytes) 000310[0] 1736261598.043782 mapwrite 0x48df24 thru 0x48f174 (0x1251 bytes) 000311[1] 1736261598.077915 mapread 0x533cd1 thru 0x535129 (0x1459 bytes) 000312[0] 1736261598.091499 mapwrite 0x7bade3 thru 0x7bc3fd (0x161b bytes) 000316[1] 1736261598.122226 mapwrite 0x30b9e6 thru 0x30c093 (0x6ae bytes) 000317[1] 1736261598.146091 read 0x6db2b5 thru 0x6db3c0 (0x10c bytes) 000321[0] 1736261598.162907 read 0x251dda thru 0x252502 (0x729 bytes) 000330[1] 1736261598.207480 mapread 0x70e187 thru 0x70f5d0 (0x144a bytes) 000331[1] 1736261598.212453 read 0x527e4f thru 0x5297c0 (0x1972 bytes) 000334[1] 1736261598.231208 mapwrite 0x61201e thru 0x6125e8 (0x5cb bytes) 000335[0] 1736261598.261936 mapread 0x564d23 thru 0x565824 (0xb02 bytes) 000340[1] 1736261598.292169 trunc from 0x7bc3fe to 0x0ce175 (0x6ee288 bytes) 000341[1] 1736261598.300661 mapwrite 0x907c45 thru 0x909755 (0x1b11 bytes) 000343[1] 1736261598.329491 read 0x830711 thru 0x8308c4 (0x1b4 bytes) 000344[0] 1736261598.334115 trunc from 0x909756 to 0x9a4893 (0x9b13e bytes) 000346[0] 1736261598.348162 trunc from 0x9a4894 to 0x3366c4 (0x66e1cf bytes) 000348[1] 1736261598.360267 mapwrite 0x49cf60 thru 0x49da2a (0xacb bytes) 000349[0] 1736261598.382926 write 0x69974b thru 0x699a80 (0x336 bytes) 000350[0] 1736261598.388544 trunc from 0x699a81 to 0x457e7a (0x241c06 bytes) 000357[1] 1736261598.421951 mapwrite 0x96ee7c thru 0x970727 (0x18ac bytes) 000358[0] 1736261598.453380 mapread 0x5f14aa thru 0x5f2047 (0xb9e bytes) 000359[0] 1736261598.462407 mapread 0x49099a thru 0x491021 (0x688 bytes) 000360[0] 1736261598.486037 write 0x0ac6ba thru 0x0ae3bf (0x1d06 bytes) 000361[0] 1736261598.492517 mapread 0x5290c8 thru 0x52af37 (0x1e70 bytes) 000362[0] 1736261598.493010 read 0x5bb101 thru 0x5bc235 (0x1135 bytes) 000363[1] 1736261598.496255 write 0x82cb1c thru 0x82d221 (0x706 bytes) 000364[1] 1736261598.514321 write 0x279979 thru 0x27a380 (0xa08 bytes) 000365[1] 1736261598.517048 mapwrite 0x1b0da4 thru 0x1b1845 (0xaa2 bytes) 000366[0] 1736261598.546087 mapread 0x3f8ccd thru 0x3f9c65 (0xf99 bytes) 000367[0] 1736261598.555116 write 0x2704a5 thru 0x2722ec (0x1e48 bytes) 000369[1] 1736261598.568515 mapread 0x138746 thru 0x13a4d0 (0x1d8b bytes) 000371[0] 1736261598.578157 read 0x377ea1 thru 0x379279 (0x13d9 bytes) 000372[1] 1736261598.604345 trunc from 0x970728 to 0x1dc695 (0x794092 bytes) 000373[0] 1736261598.623521 write 0x2145a8 thru 0x215abe (0x1517 bytes) 000375[0] 1736261598.635035 mapread 0x115800 thru 0x115eaa (0x6ab bytes) 000379[1] 1736261598.640841 trunc from 0x215abf to 0x2f9d87 (0xe42c9 bytes) 000380[0] 1736261598.650196 mapwrite 0x0dd899 thru 0x0df106 (0x186e bytes) 000381[0] 1736261598.675985 trunc from 0x2f9d88 to 0x34209b (0x48314 bytes) 000382[1] 1736261598.682547 trunc from 0x34209c to 0x04e6bb (0x2f39e0 bytes) 000385[0] 1736261598.700568 write 0x4da56f thru 0x4dbe84 (0x1916 bytes) 000392[0] 1736261598.728058 read 0x2274c0 thru 0x22854d (0x108e bytes) 000393[1] 1736261598.738539 mapread 0x4b1f13 thru 0x4b3ac7 (0x1bb5 bytes) 000396[1] 1736261598.761682 mapread 0x364638 thru 0x365ab9 (0x1482 bytes) 000398[1] 1736261598.769142 read 0x0bba48 thru 0x0bc7f9 (0xdb2 bytes) 000399[0] 1736261598.785734 mapwrite 0x604af1 thru 0x605875 (0xd85 bytes) 000400[1] 1736261598.821050 mapwrite 0x71b449 thru 0x71cff1 (0x1ba9 bytes) 000402[0] 1736261598.872520 read 0x6e18f7 thru 0x6e1f78 (0x682 bytes) 000405[0] 1736261598.884099 read 0x4081b6 thru 0x40835d (0x1a8 bytes) 000410[1] 1736261598.899597 mapread 0x29847a thru 0x2991d8 (0xd5f bytes) 000413[0] 1736261598.909435 read 0x667eb8 thru 0x6695c4 (0x170d bytes) 000414[0] 1736261598.914766 mapread 0x183914 thru 0x1853a1 (0x1a8e bytes) 000417[0] 1736261598.930238 trunc from 0x71cff2 to 0x1d415f (0x548e92 bytes) 000420[0] 1736261598.948313 trunc from 0x1d4160 to 0x5ada66 (0x3d9907 bytes) 000421[1] 1736261598.961080 mapwrite 0x6f3dfb thru 0x6f503d (0x1243 bytes) 000422[1] 1736261598.995858 mapread 0x39ffb9 thru 0x3a0636 (0x67e bytes) 000423[1] 1736261598.999405 mapread 0x5b8723 thru 0x5b937f (0xc5d bytes) 000424[0] 1736261599.005379 read 0x6a313c thru 0x6a3395 (0x25a bytes) 000425[1] 1736261599.014573 mapread 0x3a789f thru 0x3a92b2 (0x1a14 bytes) 000426[0] 1736261599.022054 write 0x1bd21e thru 0x1be982 (0x1765 bytes) 000428[0] 1736261599.030710 read 0x520a2c thru 0x521bb4 (0x1189 bytes) 000435[1] 1736261599.060313 read 0x03f73b thru 0x03f74e (0x14 bytes) 000437[1] 1736261599.069844 mapread 0x13a4e2 thru 0x13c05e (0x1b7d bytes) 000438[0] 1736261599.104261 trunc from 0x6f503e to 0x6e3db9 (0x11284 bytes) 000439[0] 1736261599.123655 write 0x34c9ad thru 0x34cbbf (0x213 bytes) 000440[1] 1736261599.128837 write 0x91f114 thru 0x91fbfb (0xae8 bytes) 000441[1] 1736261599.136633 mapwrite 0x1b3925 thru 0x1b3dfb (0x4d7 bytes) 000442[1] 1736261599.155903 mapread 0x52e4e1 thru 0x52f461 (0xf81 bytes) 000443[0] 1736261599.161717 trunc from 0x91fbfc to 0x65212e (0x2cdacd bytes) 000444[1] 1736261599.168750 read 0x46c316 thru 0x46c702 (0x3ed bytes) 000446[1] 1736261599.181883 write 0x566f28 thru 0x568d74 (0x1e4d bytes) 000447[0] 1736261599.192459 read 0x0ced56 thru 0x0d0cd6 (0x1f81 bytes) 000448[0] 1736261599.200319 write 0x1e35f9 thru 0x1e3d67 (0x76f bytes) 000450[1] 1736261599.236357 mapwrite 0x9c5e1e thru 0x9c705b (0x123e bytes) 000451[0] 1736261599.290821 mapwrite 0x612a08 thru 0x61367e (0xc77 bytes) 000453[1] 1736261599.331325 mapread 0x263964 thru 0x264600 (0xc9d bytes) 000454[1] 1736261599.342412 write 0x63e01b thru 0x63f3ea (0x13d0 bytes) 000456[0] 1736261599.351252 trunc from 0x9c705c to 0x1f8890 (0x7ce7cb bytes) truncating to largest ever: 0x9effb3 000458[1] 1736261599.374152 trunc from 0x1f8891 to 0x9effb2 (0x7f7722 bytes) 000462[1] 1736261599.403292 write 0x927e58 thru 0x928759 (0x902 bytes) 000463[0] 1736261599.409567 write 0x92d13d thru 0x92d454 (0x318 bytes) 000466[1] 1736261599.430846 mapwrite 0x360b67 thru 0x361d36 (0x11d0 bytes) 000467[0] 1736261599.456500 mapread 0x1aa788 thru 0x1aace8 (0x561 bytes) 000469[0] 1736261599.469905 read 0x125d11 thru 0x127423 (0x1713 bytes) 000470[1] 1736261599.480961 trunc from 0x9effb3 to 0x32c4c1 (0x6c3af1 bytes) 000471[0] 1736261599.490126 write 0x48d009 thru 0x48e45b (0x1453 bytes) 000473[0] 1736261599.500194 mapwrite 0x3b6061 thru 0x3b6aff (0xa9f bytes) 000474[0] 1736261599.538166 write 0x4bac3b thru 0x4bcbbe (0x1f84 bytes) 000478[0] 1736261599.553297 mapread 0x00ff90 thru 0x01109c (0x110d bytes) 000479[1] 1736261599.559755 mapwrite 0x14c568 thru 0x14c844 (0x2dd bytes) 000481[1] 1736261599.596287 mapread 0x15760d thru 0x15880c (0x1200 bytes) 000482[0] 1736261599.605416 mapread 0x0e6daa thru 0x0e8572 (0x17c9 bytes) 000484[1] 1736261599.619614 mapwrite 0x2ac110 thru 0x2acaeb (0x9dc bytes) 000489[1] 1736261599.656298 write 0x941d11 thru 0x942888 (0xb78 bytes) 000491[1] 1736261599.666944 mapread 0x8f2c99 thru 0x8f4182 (0x14ea bytes) 000494[0] 1736261599.672014 read 0x7d788d thru 0x7d8ad5 (0x1249 bytes) 000497[1] 1736261599.680993 mapread 0x798c35 thru 0x79a1e3 (0x15af bytes) 000500[1] 1736261599.695090 mapwrite 0x13fe21 thru 0x1402b8 (0x498 bytes) All operations completed A-OK! PASS 16b (7s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 09:53:20 (1736261600) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 09:53:21 (1736261601) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0881759 s, 23.8 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0686292 s, 30.6 MB/s ldlm.namespaces.MGC192.168.203.121@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b58af000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b63d9800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b58af000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b63d9800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b63d9800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 3.21727 s, 32.6 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.894973 s, 117 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.676071 s, 155 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.54791 s, 67.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.42473 s, 43.2 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.00123 s, 105 MB/s PASS 16d (14s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 09:53:35 (1736261615) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0666969 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0355101 s, 29.5 MB/s PASS 16e (1s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 09:53:36 (1736261616) TIMEOUT PASS 16f (22s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 09:53:58 (1736261638) TIMEOUT PASS 16g (22s) == sanityn test 16i: read after truncate file ============ 09:54:20 (1736261660) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (2s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 09:54:22 (1736261662) fail_loc=0x8000030a PASS 17 (3s) == sanityn test 18: mmap sanity check =========================================================================================== 09:54:25 (1736261665) mmap test1: basic mmap operation (PASS, 0.015132s) mmap test2: MAP_PRIVATE not write back (PASS, 0.01146s) mmap test3: concurrent mmap ops on two nodes (PASS, 2.3751s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.0673s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.021874s) mmap test6: check mmap write/read content on two nodes (PASS, 0.027378s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0122s) PASS 18 (8s) == sanityn test 19: test concurrent uncached read races ========================================================================= 09:54:33 (1736261673) oleg321-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg321-client: oleg321-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 09:54:34 (1736261674) PASS 20 (2s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 09:54:36 (1736261676) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (1s) == sanityn test 23: others should see updated atime while another read============================================================== 09:54:37 (1736261677) now is 1736261678 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6848 new atime is 1736261739 PASS 23 (63s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 09:55:40 (1736261740) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 374 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 375 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 374 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 375 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 8.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 5.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 13.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117974 374 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118071 375 117696 1% /mnt/lustre2[OST:1] filesystem_summary: 235665 369 235296 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 374 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 375 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 117974 374 117600 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118071 375 117696 1% /mnt/lustre[OST:1] filesystem_summary: 235665 369 235296 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117974 374 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118071 375 117696 1% /mnt/lustre2[OST:1] filesystem_summary: 235665 369 235296 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre2 PASS 24a (1s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 09:55:41 (1736261741) PASS 24b (2s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 09:55:43 (1736261743) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (1s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 09:55:44 (1736261744) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 09:55:45 (1736261745) utime: good mknod times 1736261744 <= 1736261745 <= 1736261745 for /mnt/lustre/f26a utime: good mknod times 1736261744 <= 1736261745 <= 1736261745 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (2s) == sanityn test 26b: sync mtime between ost and mds ====== 09:55:47 (1736261747) PASS 26b (3s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 09:55:51 (1736261751) 0 PASS 30 (4s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 09:55:55 (1736261755) fail_loc=0x314 PASS 31a (2s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 09:55:57 (1736261757) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg321-server mds-ost sync done. sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.203.121@tcp:/lustre 7542784 13312 7523328 1% /mnt/lustre2 PASS 31b (20s) == sanityn test 31r: open-rename(replace) race =========== 09:56:17 (1736261777) -rw-r--r-- 1 root root 0 Jan 7 09:56 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 mv: cannot stat '/mnt/lustre2/f31r.sanityn.source': Input/output error PASS 31r (5s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 09:56:22 (1736261782) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 09:56:23 (1736261783) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (0s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 09:56:23 (1736261783) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 09:56:24 (1736261784) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 09:56:25 (1736261785) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg321-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b58af000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b63d9800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800b58af000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8800b63d9800.ost_server_uuid in FULL state after 0 sec oleg321-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b58af000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b63d9800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800b58af000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800b63d9800.ost_server_uuid in IDLE state after 0 sec oleg321-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b58af000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b63d9800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800b58af000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8800b63d9800.ost_server_uuid in FULL state after 0 sec oleg321-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b58af000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b63d9800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800b58af000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8800b63d9800.ost_server_uuid in IDLE state after 0 sec oleg321-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800b58af000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b63d9800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800b58af000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8800b63d9800.ost_server_uuid in FULL state after 0 sec oleg321-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800b58af000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b63d9800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800b58af000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8800b63d9800.ost_server_uuid in IDLE state after 0 sec PASS 34 (70s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 09:57:35 (1736261855) fail_loc=0x80000317 Race attempt 0 Wait for 15618 15761 for 60 sec... total: 4000 open/close in 7.89 seconds: 507.16 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.203.121@tcp:/lustre 7.2G 15M 7.2G 1% /mnt/lustre 192.168.203.121@tcp:/lustre 7.2G 15M 7.2G 1% /mnt/lustre2 PASS 35 (64s) == sanityn test 36: handle ESTALE/open-unlink correctly == 09:58:39 (1736261919) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg321-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.287339 s, 182 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6848 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.262682 s, 200 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6848 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.24682 s, 212 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6848 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.261616 s, 200 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6848 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.240467 s, 218 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6848 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.272401 s, 192 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6848 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.248512 s, 211 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6848 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.258826 s, 203 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6848 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.311439 s, 168 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6848 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.278176 s, 188 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6848 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.31584 s, 166 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6848 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7523328) after_dd(7472128) after(7523328) PASS 36 (228s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 10:02:27 (1736262147) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6848 total: 10000 create in 9.54 seconds: 1048.21 ops/second PASS 37 (12s) == sanityn test 39a: test from 11063 ============================================================================================ 10:02:39 (1736262159) repeat after cancel_lru_locks PASS 39a (1s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 10:02:40 (1736262160) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 10:02:43 (1736262163) repeat after cancel_lru_locks PASS 39c (2s) == sanityn test 39d: sync write should update mtime ====== 10:02:45 (1736262165) fail_loc=0x411 fail_loc=0 PASS 39d (2s) == sanityn test 40a: pdirops: create vs others ======================================================================== 10:02:47 (1736262167) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (7s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 10:02:54 (1736262174) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (7s) == sanityn test 40c: pdirops: link and others ======================================================================== 10:03:01 (1736262181) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (6s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 10:03:07 (1736262187) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (7s) == sanityn test 40e: pdirops: rename and others ======================================================================== 10:03:14 (1736262194) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (7s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 10:03:21 (1736262201) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (4s) == sanityn test 41b: pdirops: create vs create ======================================================================== 10:03:25 (1736262205) open: File exists Conflict PASS 41b (4s) == sanityn test 41c: pdirops: create vs link ======================================================================== 10:03:29 (1736262209) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (4s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 10:03:33 (1736262213) Conflict PASS 41d (4s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 10:03:37 (1736262217) Conflict PASS 41e (4s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 10:03:41 (1736262221) Conflict PASS 41f (4s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 10:03:45 (1736262225) Conflict PASS 41g (4s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 10:03:49 (1736262229) Conflict PASS 41h (4s) == sanityn test 41i: reint_open: create vs create ======== 10:03:53 (1736262233) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (650s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 10:14:43 (1736262883) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (8s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 10:14:51 (1736262891) open: File exists Conflict PASS 42b (5s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 10:14:56 (1736262896) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (5s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 10:15:01 (1736262901) Conflict PASS 42d (5s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 10:15:06 (1736262906) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (4s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 10:15:10 (1736262910) Conflict PASS 42f (5s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 10:15:15 (1736262915) Conflict PASS 42g (5s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 10:15:20 (1736262920) Conflict PASS 42h (5s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 10:15:25 (1736262925) PASS 43a (16s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 10:15:41 (1736262941) Conflict PASS 43b (5s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 10:15:46 (1736262946) Conflict PASS 43c (4s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 10:15:50 (1736262950) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (5s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 10:15:55 (1736262955) Conflict PASS 43e (4s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 10:15:59 (1736262959) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (4s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 10:16:03 (1736262963) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (4s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 10:16:07 (1736262967) Conflict PASS 43h (5s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 10:16:12 (1736262972) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 10:16:13 (1736262973) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (34s) == sanityn test 43k: unlink vs create ==================== 10:16:47 (1736263007) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (428s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 10:23:56 (1736263436) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Tue Jan 7 10:24:00 EST 2025 PASS 44a (7s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 10:24:02 (1736263442) open: File exists Conflict PASS 44b (6s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 10:24:08 (1736263448) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (6s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 10:24:14 (1736263454) Conflict PASS 44d (5s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 10:24:19 (1736263459) Conflict PASS 44e (6s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 10:24:25 (1736263465) Conflict PASS 44f (6s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 10:24:31 (1736263471) Conflict PASS 44g (5s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 10:24:37 (1736263477) Conflict PASS 44h (5s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 10:24:42 (1736263482) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 10:24:43 (1736263483) PASS 45a (36s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 10:25:19 (1736263519) Conflict PASS 45b (6s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 10:25:25 (1736263525) Conflict PASS 45c (6s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 10:25:31 (1736263531) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (6s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 10:25:37 (1736263537) Conflict PASS 45e (6s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 10:25:43 (1736263543) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (6s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 10:25:49 (1736263549) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (6s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 10:25:55 (1736263555) Conflict PASS 45h (4s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 10:26:00 (1736263560) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (2s) == sanityn test 45j: read vs rename ====================== 10:26:01 (1736263561) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (383s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 10:32:24 (1736263944) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (5s) == sanityn test 46b: pdirops: link vs create ======================================================================== 10:32:29 (1736263949) open: File exists Conflict PASS 46b (5s) == sanityn test 46c: pdirops: link vs link ======================================================================== 10:32:34 (1736263954) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (4s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 10:32:38 (1736263958) Conflict PASS 46d (5s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 10:32:43 (1736263963) Conflict PASS 46e (4s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 10:32:47 (1736263967) Conflict PASS 46f (5s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 10:32:52 (1736263972) Conflict PASS 46g (4s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 10:32:56 (1736263976) Conflict PASS 46h (5s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 10:33:01 (1736263981) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (0s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 10:33:01 (1736263981) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 10:33:02 (1736263982) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 10:33:03 (1736263983) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 10:33:04 (1736263984) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 10:33:05 (1736263985) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (0s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 10:33:05 (1736263985) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 10:33:06 (1736263986) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 10:33:07 (1736263987) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.00345756 s, 3.0 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 10:33:14 (1736263994) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00340229 s, 46.7 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (24764) - No such process PASS 51a (3s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 10:33:17 (1736263997) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00349278 s, 293 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154847 s, 67.7 MB/s 3145728 PASS 51b (14s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 10:33:31 (1736264011) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00278072 s, 368 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 10:33:37 (1736264017) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983049 s, 107 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (2s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 10:33:39 (1736264019) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2434 0x982 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2434 0x982 0 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 10:33:43 (1736264023) llite.lustre-ffff8800b58af000.xattr_cache=1 llite.lustre-ffff8800b63d9800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504963 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 10:33:43.000000000 -0500 Modify: 2025-01-07 10:33:43.000000000 -0500 Change: 2025-01-07 10:33:43.000000000 -0500 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504966 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 10:33:48.000000000 -0500 Modify: 2025-01-07 10:33:49.000000000 -0500 Change: 2025-01-07 10:33:49.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504969 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 10:33:54.000000000 -0500 Modify: 2025-01-07 10:33:54.000000000 -0500 Change: 2025-01-07 10:33:54.000000000 -0500 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 10:33:54.000000000 -0500 Modify: 2025-01-07 10:33:54.000000000 -0500 Change: 2025-01-07 10:33:54.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504972 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 10:33:59.000000000 -0500 Modify: 2025-01-07 10:33:59.000000000 -0500 Change: 2025-01-07 10:33:59.000000000 -0500 Birth: - File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 10:33:59.000000000 -0500 Modify: 2025-01-07 10:33:59.000000000 -0500 Change: 2025-01-07 10:33:59.000000000 -0500 Birth: - PASS 54 (22s) == sanityn test 55a: rename vs unlink target dir ========= 10:34:05 (1736264045) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 10:34:12 (1736264052) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 10:34:19 (1736264059) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 10:34:31 (1736264071) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) == sanityn test 60: Verify data_version behaviour ======== 10:34:40 (1736264080) PASS 60 (1s) == sanityn test 70a: cd directory && rm directory ======== 10:34:41 (1736264081) PASS 70a (2s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 10:34:43 (1736264083) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 10:34:45 (1736264085) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00213581 s, 30.7 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (0s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 10:34:45 (1736264085) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.0035023 s, 18.7 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 10:34:46 (1736264086) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 10:34:47 (1736264087) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00300203 s, 21.8 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 10:34:48 (1736264088) llite.lustre-ffff8800b58af000.xattr_cache=1 llite.lustre-ffff8800b63d9800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (1s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 10:34:49 (1736264089) llite.lustre-ffff8800b58af000.xattr_cache=1 llite.lustre-ffff8800b63d9800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (2s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 10:34:51 (1736264091) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00276027 s, 371 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00249556 s, 410 kB/s 3407: get lock2 3408: get lock1 3407: done 3407: get lock1 3408: done 3408: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 3407: cannot get lock1: Resource deadlock avoided fcntl cmd 7 failed: Resource deadlock avoided 3408: cannot get lock2: Resource deadlock avoided 3408: exit rc=0 3407: done 3407: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 10:34:57 (1736264097) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0161484 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0087821 s, 119 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 4043 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (6s) == sanityn test 76: Verify MDT open_files listing ======== 10:35:03 (1736264103) opening files: ........ closing files: ........ PASS 76 (14s) == sanityn test 77a: check FIFO NRS policy =============== 10:35:17 (1736264117) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0763767 s, 220 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00724021 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100232 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013889 s, 75.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236526 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025116 s, 41.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227554 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288883 s, 36.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0275269 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234579 s, 44.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0387065 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293724 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304763 s, 34.4 MB/s , 0.0286564 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398995 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395671 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333771 s, 31.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b58af000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00884027 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00792371 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00137509 s, 763 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00305995 s, 343 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00865153 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00823321 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0090032 s, 116 MB/s 1048576 bytes (1.0 MB) copied, 0.00799923 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00354527 s, 296 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00952312 s, 110 MB/s 1048576 bytes (1.0 MB) copied, 0.0017374 s, 604 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105738 s, 99.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00964464 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0094888 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128627 s, 81.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114862 s, 91.3 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg321-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) == sanityn test 77b: check CRR-N NRS policy ============== 10:35:19 (1736264119) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0883757 s, 190 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0073047 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00903579 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00625518 s, 168 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159802 s, 65.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174847 s, 60.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251246 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00617858 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261745 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126909 s, 82.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250977 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00651303 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232032 s, 45.2 MB/s , 0.0262308 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360332 s, 29.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210989 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112952 s, 92.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00975872 s, 107 MB/s , 0.00635934 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00561022 s, 187 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00656219 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948046 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00745695 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128319 s, 81.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00772184 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0126465 s, 82.9 MB/s 1048576 bytes (1.0 MB) copied, 0.00154175 s, 680 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00962045 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110843 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105992 s, 98.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116489 s, 90.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123878 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124295 s, 84.4 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=1 removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg321-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.107699 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00733266 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110796 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170622 s, 61.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021804 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163122 s, 64.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123048 s, 85.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0195087 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256722 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122635 s, 85.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315031 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352272 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125287 s, 83.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0204907 s, 51.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.024027 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281809 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315437 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00999909 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00670979 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00289483 s, 362 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00797526 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00372793 s, 281 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0045027 s, 233 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00868891 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00940015 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00950582 s, 110 MB/s , 0.00866227 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00856378 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00615279 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00935482 s, 112 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0109986 s, 95.3 MB/s , 0.0108585 s, 96.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112 s, 93.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=1 removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg321-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) == sanityn test 77c: check ORR NRS policy ================ 10:35:22 (1736264122) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.105936 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00767883 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184408 s, 56.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183731 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119886 s, 87.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149252 s, 70.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173447 s, 60.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185358 s, 56.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268018 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144809 s, 72.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163016 s, 64.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261496 s, 40.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0285391 s, 36.7 MB/s , 0.0272823 s, 38.4 MB/s 1048576 bytes (1.0 MB) copied, 0.016875 s, 62.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213692 s, 49.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972821 s, 108 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b58af000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00618775 s, 169 MB/s , 0.00781875 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00819226 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00670714 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00822396 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106223 s, 98.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00162738 s, 644 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00837071 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00928321 s, 113 MB/s 1048576 bytes (1.0 MB) copied, 0.0083369 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00916989 s, 114 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00823547 s, 127 MB/s , 0.00872053 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00891101 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918461 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132931 s, 78.9 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=1 removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg321-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0876335 s, 191 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0084492 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128567 s, 81.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109058 s, 96.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0201776 s, 52.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175247 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027103 s, 38.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281348 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161894 s, 64.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0243103 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199911 s, 52.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153151 s, 68.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265894 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00943943 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182875 s, 57.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0275324 s, 38.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b58af000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0287143 s, 36.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.011721 s, 89.5 MB/s , 0.00756717 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00714344 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00801263 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00898873 s, 117 MB/s 1048576 bytes (1.0 MB) copied, 0.00880895 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135802 s, 77.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00938184 s, 112 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111879 s, 93.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124347 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125073 s, 83.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115631 s, 90.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134199 s, 78.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01397 s, 75.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116149 s, 90.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144694 s, 72.5 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg321-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0851977 s, 197 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00849392 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132699 s, 79.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228042 s, 46.0 MB/s , 0.0284148 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293149 s, 35.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284427 s, 36.9 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0328353 s, 31.9 MB/s , 0.0260258 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250105 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0252703 s, 41.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245956 s, 42.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0198602 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245742 s, 42.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154221 s, 68.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200768 s, 52.2 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231947 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00887189 s, 118 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0016177 s, 648 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00162315 s, 646 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00734777 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00146148 s, 717 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113755 s, 92.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0106601 s, 98.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00298481 s, 351 MB/s , 0.00223091 s, 470 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118386 s, 88.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122546 s, 85.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00154996 s, 677 MB/s 1048576 bytes (1.0 MB) copied, 0.00384772 s, 273 MB/s 1048576 bytes (1.0 MB) copied, 0.00959204 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139965 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00144222 s, 727 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg321-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (4s) == sanityn test 77d: check TRR nrs policy ================ 10:35:26 (1736264126) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.120911 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00911188 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0092302 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268782 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198533 s, 52.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0358586 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.036422 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317456 s, 33.0 MB/s , 0.0396498 s, 26.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0336926 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0438783 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356638 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327478 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0418322 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0339218 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0422077 s, 24.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b58af000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.043582 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00161906 s, 648 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00711544 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00751478 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00250209 s, 419 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109543 s, 95.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00971082 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00995989 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00259883 s, 403 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00135973 s, 771 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011374 s, 92.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00280329 s, 374 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151701 s, 69.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133928 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132034 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127128 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143433 s, 73.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=2 removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg321-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.121668 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00815682 s, 129 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0214759 s, 48.8 MB/s , 0.0156347 s, 67.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0202463 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155216 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022972 s, 45.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286494 s, 36.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0295959 s, 35.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306701 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0399428 s, 26.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b58af000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0396561 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0428026 s, 24.5 MB/s 1+0 records in 1+0 records out , 0.0338503 s, 31.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0404244 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0420324 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0435888 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100479 s, 104 MB/s , 0.00832921 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104673 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00685006 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00871415 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00376712 s, 278 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00842767 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00916125 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00142127 s, 738 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143399 s, 73.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00887938 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00992112 s, 106 MB/s , 0.00307755 s, 341 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00953319 s, 110 MB/s 1048576 bytes (1.0 MB) copied, 0.00913286 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113016 s, 92.8 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=3 removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg321-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0905584 s, 185 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00795134 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00898608 s, 117 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0163549 s, 64.1 MB/s , 0.0163962 s, 64.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018034 s, 58.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0155609 s, 67.4 MB/s , 0.015604 s, 67.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254354 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150866 s, 69.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251088 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190409 s, 55.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280138 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124525 s, 84.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304159 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220735 s, 47.5 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104131 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00946677 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00841336 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0055109 s, 190 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0079256 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00179943 s, 583 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00697104 s, 150 MB/s , 0.0107391 s, 97.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102984 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00131298 s, 799 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00145086 s, 723 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00158479 s, 662 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011178 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00404587 s, 259 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107414 s, 97.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107114 s, 97.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00139911 s, 749 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=1 removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg321-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (3s) == sanityn test 77e: check TBF NID nrs policy ============ 10:35:29 (1736264129) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.203.21@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0983096 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00874844 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113416 s, 92.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118776 s, 88.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193583 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00692116 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217897 s, 48.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162112 s, 64.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.014758 s, 71.1 MB/s , 0.0180502 s, 58.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00699403 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225076 s, 46.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0370823 s, 28.3 MB/s , 0.0352901 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259795 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327511 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319714 s, 32.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00659003 s, 159 MB/s , 0.00663723 s, 158 MB/s 1048576 bytes (1.0 MB) copied, 0.00889366 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00643202 s, 163 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00736838 s, 142 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00744065 s, 141 MB/s , 0.00890083 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00841937 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0088569 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0086549 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00906558 s, 116 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00911089 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00847533 s, 124 MB/s 1048576 bytes (1.0 MB) copied, 0.0101089 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0090455 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0091509 s, 115 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg321-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0785504 s, 214 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00613922 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00893281 s, 117 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016211 s, 64.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177437 s, 59.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200217 s, 52.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0137385 s, 76.3 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0134792 s, 77.8 MB/s , 0.0203949 s, 51.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239857 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0057157 s, 183 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026379 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195533 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195483 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103048 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210908 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00709025 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00687599 s, 152 MB/s 1048576 bytes (1.0 MB) copied, 0.00693867 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00794695 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00926836 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0074462 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00723372 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0066555 s, 158 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00152092 s, 689 MB/s , 0.00151172 s, 694 MB/s , 0.00147481 s, 711 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0100798 s, 104 MB/s , 0.00907988 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00142855 s, 734 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100695 s, 104 MB/s , 0.00870412 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00125006 s, 839 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg321-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0867405 s, 193 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00781215 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139338 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147431 s, 71.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0312158 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273747 s, 38.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0283018 s, 37.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0371821 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037651 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261636 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0350487 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0325202 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289961 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279476 s, 37.5 MB/s 1048576 bytes (1.0 MB) copied, 0.034067 s, 30.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b58af000.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327416 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341649 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948803 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00860035 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00774499 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117748 s, 89.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00945229 s, 111 MB/s , 0.00984448 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00880289 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0096449 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110873 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105918 s, 99.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0112826 s, 92.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0121247 s, 86.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106374 s, 98.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011222 s, 93.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.011813 s, 88.8 MB/s , 0.0120043 s, 87.4 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg321-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0818456 s, 205 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00751587 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00802721 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196144 s, 53.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256163 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229401 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200142 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145155 s, 72.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253516 s, 41.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020552 s, 51.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0287507 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274115 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160211 s, 65.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147246 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192142 s, 54.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218132 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258742 s, 40.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b58af000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00473448 s, 221 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00867963 s, 121 MB/s , 0.00146978 s, 713 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00155371 s, 675 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00804894 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00886125 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117983 s, 88.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118806 s, 88.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00842003 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00792432 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00839065 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011248 s, 93.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00981678 s, 107 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00984751 s, 106 MB/s , 0.00966341 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00937782 s, 112 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg321-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (5s) == sanityn test 77f: check TBF JobID nrs policy ========== 10:35:35 (1736264135) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg321-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.127215 s, 132 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00777198 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103976 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116571 s, 90.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161754 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162965 s, 64.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00757882 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026123 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216773 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315549 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175822 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209257 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226034 s, 46.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207962 s, 50.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02895 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285351 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286595 s, 36.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00617252 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00638646 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00653591 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00609209 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00913828 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104307 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00712186 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00753167 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848439 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00904796 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00891356 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948484 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00958925 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00957858 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115328 s, 90.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102312 s, 102 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg321-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg321-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.110423 s, 152 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.009159 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112091 s, 93.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219207 s, 47.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174849 s, 60.0 MB/s , 0.019314 s, 54.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291324 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241681 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293661 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240898 s, 43.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354085 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378131 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294902 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0310321 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0334795 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0320416 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0409424 s, 25.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00778288 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00727648 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00933869 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103538 s, 101 MB/s 1048576 bytes (1.0 MB) copied, 0.00860515 s, 122 MB/s 1048576 bytes (1.0 MB) copied, 0.00863801 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00775294 s, 135 MB/s 1048576 bytes (1.0 MB) copied, 0.00880064 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00938915 s, 112 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00976739 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00959917 s, 109 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0100555 s, 104 MB/s , 0.00986815 s, 106 MB/s 1048576 bytes (1.0 MB) copied, 0.0108181 s, 96.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104169 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011066 s, 94.8 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg321-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg321-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0877651 s, 191 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0071858 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00809095 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226122 s, 46.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0221 s, 47.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222077 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292265 s, 35.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0200073 s, 52.4 MB/s , 0.0246023 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300656 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202151 s, 51.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173117 s, 60.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0305039 s, 34.4 MB/s , 0.0179199 s, 58.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.029806 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239054 s, 43.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b58af000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295788 s, 35.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119965 s, 87.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00995358 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00759409 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00726561 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00953792 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00973813 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.009023 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984395 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112635 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117284 s, 89.4 MB/s , 0.0120643 s, 86.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0116185 s, 90.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0119671 s, 87.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013604 s, 77.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133022 s, 78.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129748 s, 80.8 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg321-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg321-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0830833 s, 202 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00804804 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00977901 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183315 s, 57.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286928 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290437 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190987 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022996 s, 45.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260018 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251099 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276594 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218854 s, 47.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276278 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147589 s, 71.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285185 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165336 s, 63.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copiedldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=1 , 0.0224354 s, 46.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00703254 s, 149 MB/s , 0.0018369 s, 571 MB/s , 0.00849884 s, 123 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00859892 s, 122 MB/s , 0.00874231 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0089465 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00906468 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102817 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104827 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116603 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108523 s, 96.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0123029 s, 85.2 MB/s , 0.0120604 s, 86.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120144 s, 87.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0117788 s, 89.0 MB/s , 0.0115006 s, 91.2 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=1 removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg321-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (6s) == sanityn test 77g: Change TBF type directly ============ 10:35:40 (1736264140) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (2s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 10:35:42 (1736264142) oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg321-client: oleg321-server: ssh exited with exit code 2 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg321-client: oleg321-server: ssh exited with exit code 12 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg321-client: oleg321-server: ssh exited with exit code 12 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg321-client: oleg321-server: ssh exited with exit code 12 PASS 77h (3s) == sanityn test 77i: Change rank of TBF rule ============= 10:35:45 (1736264145) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (6s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 10:35:51 (1736264151) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0815976 s, 206 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115425 s, 90.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143017 s, 73.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247252 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317398 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0325818 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292523 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284514 s, 36.9 MB/s , 0.022441 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372129 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272538 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0348127 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0366136 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345356 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0359423 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0366113 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0410781 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00994789 s, 105 MB/s 1048576 bytes (1.0 MB) copied, 0.00678022 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0091316 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00918991 s, 114 MB/s 1048576 bytes (1.0 MB) copied, 0.00776021 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00808923 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918278 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00950102 s, 110 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00992505 s, 106 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0102601 s, 102 MB/s , 0.0100079 s, 105 MB/s 1048576 bytes (1.0 MB) copied, 0.00983146 s, 107 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0108734 s, 96.4 MB/s , 0.0112725 s, 93.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112521 s, 93.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114309 s, 91.7 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg321-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.91784 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4285 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (49s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 10:36:41 (1736264201) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg321-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.640028 s, 26.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169758 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.172691 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157942 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158533 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178237 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.179341 s, 5.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.174242 s, 6.0 MB/s , 0.170894 s, 6.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.370123 s, 2.8 MB/s , 0.371018 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.374506 s, 2.8 MB/s , 0.360226 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.36872 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.373192 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.385486 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.368359 s, 2.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.152868 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148644 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152123 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153126 s, 6.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.149733 s, 7.0 MB/s , 0.148915 s, 7.0 MB/s , 0.150012 s, 7.0 MB/s 1048576 bytes (1.0 MB) copied, 0.15295 s, 6.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.153079 s, 6.8 MB/s , 0.151302 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150602 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150854 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151661 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.152187 s, 6.9 MB/s 1048576 bytes (1.0 MB) copied, 0.15143 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152467 s, 6.9 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg321-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8321 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9579 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg321-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.647106 s, 25.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.16433 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.162673 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.179421 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.176194 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.173889 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.187157 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178955 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.182658 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.366011 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.374802 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.37028 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.373771 s, 2.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.396088 s, 2.6 MB/s , 0.394944 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.401536 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.39915 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15171 s, 6.9 MB/s 1048576 bytes (1.0 MB) copied, 0.151781 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153127 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154009 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153242 s, 6.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152596 s, 6.9 MB/s 1048576 bytes (1.0 MB) copied, 0.152835 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154254 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157106 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154616 s, 6.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154646 s, 6.8 MB/s 1048576 bytes (1.0 MB) copied, 0.154656 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156015 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155699 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156114 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157614 s, 6.7 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg321-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8313 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9595 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (133s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 10:38:54 (1736264334) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.194032 s, 86.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013256 s, 79.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133474 s, 78.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253057 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163144 s, 64.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.012233 s, 85.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0170778 s, 61.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.029931 s, 35.0 MB/s , 0.0233856 s, 44.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0656964 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0813092 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0818533 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0634042 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0931395 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0847385 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0764886 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0854115 s, 12.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00837103 s, 125 MB/s , 0.00985759 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103144 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100997 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133489 s, 78.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00994697 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010715 s, 97.9 MB/s , 0.00987429 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123978 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120505 s, 87.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126378 s, 83.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122999 s, 85.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121139 s, 86.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119915 s, 87.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136537 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131598 s, 79.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92448 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73048 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.21@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.21@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.1441 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985437 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027036 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200719 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0330907 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.033354 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206806 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0299877 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272403 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0620074 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0693482 s, 15.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.156847 s, 6.7 MB/s , 0.165347 s, 6.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.266656 s, 3.9 MB/s , 0.254444 s, 4.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.332389 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.333148 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00781397 s, 134 MB/s 1+0 records in 1+0 records out , 0.00678754 s, 154 MB/s 1048576 bytes (1.0 MB) copied, 0.0078186 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00739103 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0099756 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111453 s, 94.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00945548 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0109755 s, 95.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0109928 s, 95.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108387 s, 96.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115303 s, 90.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011972 s, 87.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126277 s, 83.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125244 s, 83.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122724 s, 85.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112486 s, 93.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92547 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72756 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.21@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.154228 s, 109 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109611 s, 95.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139274 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0305241 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280515 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0287753 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236101 s, 44.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0212972 s, 49.2 MB/s , 0.0305605 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0587931 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0561541 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0708886 s, 14.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0805069 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0676739 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0722084 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0881909 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0904768 s, 11.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00906613 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0069136 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00744128 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00818684 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101095 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118594 s, 88.4 MB/s , 0.0128995 s, 81.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104883 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117366 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0118158 s, 88.7 MB/s 1048576 bytes (1.0 MB) copied, 0.011669 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108146 s, 97.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121719 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00999903 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126169 s, 83.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134456 s, 78.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93569 s, 21.2 MB/s Write runtime is 5 s, speed is 20.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.88008 s, 21.5 MB/s Read runtime is 5 s, speed is 20.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.252198 s, 66.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181075 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341306 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0350066 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03381 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398725 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283121 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215122 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0264583 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153704 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150135 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.312584 s, 3.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.316673 s, 3.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.347187 s, 3.0 MB/s , 0.360056 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.362099 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.356455 s, 2.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00887244 s, 118 MB/s , 0.00982097 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108623 s, 96.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104887 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00989991 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106263 s, 98.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106343 s, 98.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110747 s, 94.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113781 s, 92.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011498 s, 91.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120458 s, 87.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114891 s, 91.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0128404 s, 81.7 MB/s , 0.0130714 s, 80.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122698 s, 85.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159131 s, 65.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg321-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.76699 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73105 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.91175 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72812 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.644387 s, 26.0 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166279 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169266 s, 6.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.16024 s, 6.5 MB/s , 0.166525 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.179771 s, 5.8 MB/s 1048576 bytes (1.0 MB) copied, 0.170687 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.171572 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.165358 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.371963 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.377527 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.38035 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.382871 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.393701 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.383575 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.396666 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.399194 s, 2.6 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00946153 s, 111 MB/s , 0.0119784 s, 87.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00995644 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00955879 s, 110 MB/s , 0.0103626 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110824 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114072 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113902 s, 92.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114812 s, 91.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135883 s, 77.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0136484 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129833 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130589 s, 80.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014058 s, 74.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143802 s, 72.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154377 s, 67.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg321-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8267 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4348 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.149528 s, 112 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021676 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215317 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259986 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306675 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327811 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0430241 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346163 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.029688 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.05823 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0625913 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0619928 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0659441 s, 15.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0658234 s, 15.9 MB/s , 0.0714005 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0672154 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.063476 s, 16.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00827466 s, 127 MB/s , 0.00630766 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00514074 s, 204 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00695768 s, 151 MB/s , 0.00931506 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103367 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00909626 s, 115 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00892966 s, 117 MB/s , 0.00881755 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00909827 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106191 s, 98.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112754 s, 93.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.010953 s, 95.7 MB/s , 0.0113064 s, 92.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120882 s, 86.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119728 s, 87.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg321-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93214 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73246 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.245393 s, 68.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00768325 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226067 s, 46.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021898 s, 47.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175746 s, 59.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193849 s, 54.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186149 s, 56.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0267714 s, 39.2 MB/s , 0.030526 s, 34.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.168439 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178974 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.17865 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.184878 s, 5.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.324019 s, 3.2 MB/s , 0.315472 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.358105 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.360853 s, 2.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00567504 s, 185 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00664659 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00819499 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00758963 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102031 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00812956 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00851393 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00907121 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00912077 s, 115 MB/s , 0.00879761 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00816207 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0098319 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101754 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00935899 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109203 s, 96.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010884 s, 96.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg321-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77617 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72798 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.76295 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72703 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (350s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 10:44:45 (1736264685) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (2s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 10:44:47 (1736264687) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg321-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.1006 s, 338 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg321-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0151 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (49s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 10:45:36 (1736264736) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0936134 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150713 s, 69.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167357 s, 62.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01929 s, 54.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176063 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255827 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0323109 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386745 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03446 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0422857 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0443993 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324683 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0462127 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335402 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331222 s, 31.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b58af000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276983 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0454828 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01167 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00155184 s, 676 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108822 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010237 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150355 s, 69.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0131837 s, 79.5 MB/s , 0.0116209 s, 90.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112268 s, 93.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138895 s, 75.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127054 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00140208 s, 748 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134487 s, 78.0 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff8800b58af000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b58af000.lock_unused_count=1 1048576 bytes (1.0 MB) copied, 0.0136738 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0015681 s, 669 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014072 s, 74.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124065 s, 84.5 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg321-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92818 s, 21.3 MB/s Write runtime is 5 s, speed is 20.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96799 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.187201 s, 89.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123612 s, 84.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171995 s, 61.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101908 s, 103 MB/s 1048576 bytes (1.0 MB) copied, 0.0125525 s, 83.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212709 s, 49.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00631532 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00688545 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0097056 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0711624 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0767585 s, 13.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0865701 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0789393 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.168222 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.162575 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.264188 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.24672 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189633 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210714 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191823 s, 54.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209746 s, 50.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.020232 s, 51.8 MB/s , 0.0202527 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191008 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203842 s, 51.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196933 s, 53.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206326 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220528 s, 47.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216085 s, 48.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0218145 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219142 s, 47.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219709 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237903 s, 44.1 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg321-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg321-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.9313 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96562 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (64s) == sanityn test 77o: Changing rank should not panic ====== 10:46:40 (1736264800) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (3s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 10:46:43 (1736264803) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (25s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 10:47:08 (1736264828) mds.MDS.mdt.nrs_policies=tbf oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg321-client: oleg321-server: ssh exited with exit code 36 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg321-client: oleg321-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (9s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 10:47:17 (1736264837) oleg321-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device pdsh@oleg321-client: oleg321-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (2s) == sanityn test 79: xattr: intent error ================== 10:47:19 (1736264839) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505431 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 10:47:20.000000000 -0500 Modify: 2025-01-07 10:47:20.000000000 -0500 Change: 2025-01-07 10:47:20.000000000 -0500 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 10:47:31 (1736264851) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 10:47:32 (1736264852) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 10:47:33 (1736264853) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 10:47:34 (1736264854) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (0s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 10:47:34 (1736264854) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 10:47:35 (1736264855) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6848 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6848 PASS 82 (2s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 10:47:37 (1736264857) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (0s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 10:47:38 (1736264858) fail_loc=0x8000060b PASS 84 (7s) == sanityn test 90: open/create and unlink striped directory ========================================================== 10:47:44 (1736264864) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 10:47:45 (1736264865) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 10:47:46 (1736264866) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 10:47:47 (1736264867) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2629 0xa45 0 0 2644 0xa54 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2645 0xa55 0 1 2630 0xa46 0 rc1=0 and rc2=0 PASS 93 (8s) == sanityn test 94: signal vs CP callback race =========== 10:47:55 (1736264875) osc.lustre-OST0000-osc-ffff8800b58af000.idle_timeout=0 osc.lustre-OST0000-osc-ffff8800b63d9800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b58af000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b63d9800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0356732 s, 230 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 2.99925 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 4844 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0159566 s, 257 kB/s osc.lustre-OST0000-osc-ffff8800b58af000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b63d9800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b58af000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b63d9800.idle_timeout=debug PASS 94 (9s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 10:48:04 (1736264884) llite.lustre-ffff8800b58af000.fast_read=0 llite.lustre-ffff8800b63d9800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00150141 s, 8.2 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800b58af000.fast_read=1 llite.lustre-ffff8800b63d9800.fast_read=1 PASS 95b (8s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 10:48:12 (1736264892) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 10:48:13 (1736264893) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00127811 s, 3.2 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (1s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 10:48:14 (1736264894) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0179971 s, 117 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 10:48:16 (1736264896) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0170026 s, 123 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (1s) == sanityn test 100e: DoM: read on open and file size ==== 10:48:17 (1736264897) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00158101 s, 7.6 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 10:48:19 (1736264899) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000543567 s, 7.5 MB/s PASS 101a (1s) == sanityn test 101b: Discard DoM data on rename ========= 10:48:20 (1736264900) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000497138 s, 8.2 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 10:48:22 (1736264902) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000608841 s, 6.7 MB/s PASS 101c (2s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 10:48:25 (1736264905) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1736264906 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1736264908 Finishing test test23 at 1736264908 PASS 103 (6s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 10:48:31 (1736264911) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 10:48:32 (1736264912) fail_loc=0x416 PASS 105 (47s) == sanityn test 106a: Verify the btime via statx() ======= 10:49:19 (1736264959) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 10:49:20 (1736264960) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) == sanityn test 106c: Verify statx attributes mask ======= 10:49:21 (1736264961) SKIP: sanityn test_106c statx() only test SKIP 106c (0s) == sanityn test 107a: Basic grouplock conflict =========== 10:49:21 (1736264961) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0650833 s, 161 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2652 0xa5c 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6848 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6848 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 10:49:25 (1736264965) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0644081 s, 163 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2634 0xa4a 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6848 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) == sanityn test 108a: lseek: parallel updates ============ 10:49:33 (1736264973) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00127882 s, 7.8 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.22416 s, 2.0 MB/s PASS 108a (6s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 10:49:39 (1736264979) 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions oleg321-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 8 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions oleg321-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 10 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions oleg321-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 16 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions oleg321-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 23 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions oleg321-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 26 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions oleg321-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 33 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions oleg321-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 42 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions oleg321-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 44 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions oleg321-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 50 Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 192.168.203.121@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.121@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg321-server' oleg321-server: oleg321-server.virtnet: executing load_modules_local oleg321-server: Loading modules from /home/green/git/lustre-release/lustre oleg321-server: detected 4 online CPUs by sysfs oleg321-server: Force libcfs to create 2 CPU partitions Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre Starting client: oleg321-client.virtnet: -o user_xattr,flock oleg321-server@tcp:/lustre /mnt/lustre2 PASS 109 (388s) == sanityn test 110: do not grant another lock on resend ========================================================== 10:56:07 (1736265367) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 10:56:07.000000000 -0500 Modify: 2025-01-07 10:56:07.000000000 -0500 Change: 2025-01-07 10:56:07.000000000 -0500 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg321-client.virtnet oleg321-client.virtnet PASS 110 (88s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 10:57:35 (1736265455) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 10:57:36 (1736265456) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 10:57:37 (1736265457) lustre-OST0000-osc-ffff88012ab49800 active. lustre-OST0001-osc-ffff88012ab49800 active. lustre-OST0000-osc-ffff88012ab49800 active. lustre-OST0001-osc-ffff88012ab49800 active. MGC192.168.203.121@tcp active. PASS 113 (2s) cleanup: ====================================================== == sanityn test complete, duration 4091 sec ============== 10:57:39 (1736265459) Stopping clients: oleg321-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg321-client.virtnet /mnt/lustre2 opts: