-----============= acceptance-small: sanityn ============----- Tue Jun 3 15:49:07 EDT 2025 oleg354-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/sanityn.*ex: No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 Started clients oleg354-client.virtnet: 192.168.203.154@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt) oleg354-client.virtnet: executing check_config_client /mnt/lustre oleg354-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg354-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff88012bf96000.idle_timeout=debug osc.lustre-OST0000-osc-ffff88012dbf1800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012bf96000.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012dbf1800.idle_timeout=debug disable quota as required oleg354-server: oleg354-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138988 s, 75.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6866] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 15:49:21 (1748980161) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (2s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 15:49:23 (1748980163) -rw-r--r-- 1 root root 0 Jun 3 15:49 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (2s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 15:49:25 (1748980165) -rw-r--r-- 1 root root 0 Jun 3 15:49 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (2s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 15:49:27 (1748980167) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (2s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 15:49:29 (1748980169) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (2s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 15:49:31 (1748980171) total 1 -rwxrwxrwx 1 root root 0 Jun 3 15:49 f2a -rwxrwxrwx 1 root root 0 Jun 3 15:49 f2b total 1 -rwxrwxrwx 1 root root 0 Jun 3 15:49 f2a -rwxrwxrwx 1 root root 0 Jun 3 15:49 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00410541 s, 125 kB/s PASS 2e (2s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 15:49:33 (1748980173) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 15:49:34 (1748980174) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.131856 s, 15.9 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (3s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 15:49:37 (1748980177) PASS 3 (2s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 15:49:39 (1748980179) PASS 4 (3s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 15:49:42 (1748980182) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (2s) == sanityn test 6: remove of open file on other node ============================================================================ 15:49:44 (1748980184) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (2s) == sanityn test 7: remove of open directory on other node ======================================================================= 15:49:46 (1748980186) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (2s) == sanityn test 8: remove of open special file on other node ==================================================================== 15:49:48 (1748980188) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (2s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 15:49:50 (1748980190) PASS 9a (2s) == sanityn test 9b: append to striped sparse file ======== 15:49:52 (1748980192) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153658 s, 68.2 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0208226 s, 0.1 kB/s Data read (expecting 'foo'): foo PASS 9b (2s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 15:49:54 (1748980194) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00247654 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00282723 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00297415 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00286476 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0028112 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00276383 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0027602 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00266317 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00283589 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0025496 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00269608 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00240851 s, 0.4 kB/s PASS 10a (3s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 15:49:57 (1748980197) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00433344 s, 709 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0106394 s, 385 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000313437 s, 9.8 MB/s PASS 10b (2s) == sanityn test 11: execution of file opened for write should return error ============================================================== 15:49:59 (1748980199) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6866 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (2s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 15:50:01 (1748980201) warning: '-runtime' deprecated, use '-t runtime' instead - link 1683 (time 1748980211.62 total 10.01 last 168.21) - link 3239 (time 1748980221.63 total 20.01 last 155.51) - link 4760 (time 1748980231.63 total 30.02 last 152.03) - link 6298 (time 1748980241.64 total 40.02 last 153.70) - link 7876 (time 1748980251.64 total 50.02 last 157.80) - link 9282 (time 1748980261.64 total 60.02 last 140.58) - link 10000 (time 1748980266.79 total 65.17 last 139.42) - link 11417 (time 1748980276.79 total 75.18 last 141.67) - link 12836 (time 1748980286.79 total 85.18 last 141.89) - link 14251 (time 1748980296.79 total 95.18 last 141.49) total: 14936 link in 100.00 seconds: 149.35 ops/second using seed 161044188 running for 100 seconds - stat 10000 (time 1748980206 ; total 4 ; last 4) - stat 20000 (time 1748980209 ; total 7 ; last 3) - stat 30000 (time 1748980213 ; total 11 ; last 4) - stat 40000 (time 1748980217 ; total 15 ; last 4) - stat 50000 (time 1748980221 ; total 19 ; last 4) - stat 60000 (time 1748980225 ; total 23 ; last 4) - stat 70000 (time 1748980229 ; total 27 ; last 4) - stat 80000 (time 1748980233 ; total 31 ; last 4) - stat 90000 (time 1748980237 ; total 35 ; last 4) - stat 100000 (time 1748980240 ; total 38 ; last 3) - stat 110000 (time 1748980244 ; total 42 ; last 4) - stat 120000 (time 1748980248 ; total 46 ; last 4) - stat 130000 (time 1748980251 ; total 49 ; last 3) - stat 139175 (time 1748980262 ; total 60 ; last 11) - stat 140000 (time 1748980262 ; total 60 ; last 0) - stat 148061 (time 1748980273 ; total 71 ; last 11) - stat 150000 (time 1748980275 ; total 73 ; last 2) - stat 157298 (time 1748980286 ; total 84 ; last 11) - stat 160000 (time 1748980289 ; total 87 ; last 3) - stat 167538 (time 1748980300 ; total 98 ; last 11) total: 169146 stats in 100 seconds: 1691.459961 stats/second - unlinked 0 (time 1748980252 ; total 0 ; last 0) - unlinked 10000 (time 1748980326 ; total 74 ; last 74) unlink(/mnt/lustre2/lockdir/lockfile14936) error: No such file or directory total: 14935 unlinks in 86 seconds: 173.662796 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (20567) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (20570) - No such process PASS 12 (138s) == sanityn test 13: test directory page revocation ======= 15:52:19 (1748980339) f13 f13 PASS 13 (1s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 15:52:20 (1748980340) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6866 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 21942 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 15:52:22 (1748980342) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4298: 22569 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (1s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 15:52:23 (1748980343) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 23196 Terminated $DIR1/$tdir/sleep 60 PASS 14b (2s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 15:52:25 (1748980345) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 23824 Terminated $DIR1/$tdir/sleep 60 PASS 14c (1s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 15:52:26 (1748980346) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 24453 Terminated $DIR1/$tdir/sleep 60 PASS 14d (2s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 15:52:28 (1748980348) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7523328 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7523328kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (11s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 15:52:39 (1748980359) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.100992 s, 20.8 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0649997 s, 32.3 MB/s Chance of close/open is 1 in 50 Seed set to 360 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000003[0] 1748980360.826326 mapwrite 0x822aa3 thru 0x82b53f (0x8a9d bytes) 000004[1] 1748980360.860529 read 0x12c471 thru 0x135ae6 (0x9676 bytes) truncating to largest ever: 0x2dc5e 000005[1] 1748980360.865209 trunc from 0x82b540 to 0x02dc5d (0x7fd8e2 bytes) 000006[0] 1748980360.875634 write 0x44c819 thru 0x45394d (0x7135 bytes) 000009[1] 1748980360.891282 write 0x7ccf34 thru 0x7dad13 (0xdde0 bytes) truncating to largest ever: 0x6522ab 000010[0] 1748980360.896475 trunc from 0x7dad14 to 0x6522aa (0x188a69 bytes) 000011[1] 1748980360.907929 trunc from 0x6522ab to 0x51bae6 (0x1367c4 bytes) 000012[0] 1748980360.915790 mapwrite 0x951e83 thru 0x95842f (0x65ad bytes) 000013[1] 1748980360.938928 mapwrite 0x4aa171 thru 0x4b4f38 (0xadc8 bytes) 000015[0] 1748980360.972912 write 0x4bbc0e thru 0x4cb936 (0xfd29 bytes) truncating to largest ever: 0x8c70ee 000017[1] 1748980360.983376 trunc from 0x958430 to 0x8c70ed (0x91342 bytes) 000022[1] 1748980361.014078 mapwrite 0x97474e thru 0x97e91c (0xa1cf bytes) 000024[1] 1748980361.041997 write 0x22e772 thru 0x23a00c (0xb89b bytes) 000026[0] 1748980361.048679 mapread 0x145460 thru 0x154041 (0xebe2 bytes) 000027[1] 1748980361.062506 mapwrite 0x1a07ec thru 0x1ab246 (0xaa5b bytes) 000030[0] 1748980361.088552 mapread 0x632121 thru 0x632179 (0x59 bytes) 000031[1] 1748980361.099173 trunc from 0x97e91d to 0x567ce3 (0x416c39 bytes) 000032[1] 1748980361.105118 write 0x9c2362 thru 0x9ccfdb (0xac7a bytes) 000033[1] 1748980361.111627 mapread 0x354ad2 thru 0x35c6a5 (0x7bd4 bytes) 000035[0] 1748980361.119997 write 0x99e555 thru 0x9a03f3 (0x1e9f bytes) 000036[1] 1748980361.129854 mapwrite 0x87b6bb thru 0x87c8fb (0x1241 bytes) 000037[0] 1748980361.156795 mapread 0x383dca thru 0x38cc3e (0x8e75 bytes) 000038[0] 1748980361.166202 mapread 0x003243 thru 0x0069a5 (0x3763 bytes) 000041[0] 1748980361.180546 write 0x2b4c9b thru 0x2b940a (0x4770 bytes) 000043[0] 1748980361.187793 read 0x49ce9c thru 0x4aae07 (0xdf6c bytes) 000047[1] 1748980361.193262 mapwrite 0x6f16ca thru 0x6f5d58 (0x468f bytes) 000049[1] 1748980361.221618 read 0x954314 thru 0x955b56 (0x1843 bytes) 000051[1] 1748980361.238231 mapwrite 0x9d3560 thru 0x9df708 (0xc1a9 bytes) 000052[0] 1748980361.261502 mapread 0x53a178 thru 0x549872 (0xf6fb bytes) 000055[0] 1748980361.281847 read 0x7e6830 thru 0x7e747f (0xc50 bytes) 000058[1] 1748980361.295356 mapwrite 0x243196 thru 0x24c152 (0x8fbd bytes) 000060[0] 1748980361.326376 mapread 0x964ee1 thru 0x966ded (0x1f0d bytes) 000061[1] 1748980361.331851 write 0x730774 thru 0x736fc4 (0x6851 bytes) 000062[1] 1748980361.335739 write 0x17a471 thru 0x17efc5 (0x4b55 bytes) 000063[1] 1748980361.341199 mapwrite 0x141cb0 thru 0x14bd05 (0xa056 bytes) 000064[0] 1748980361.367504 trunc from 0x9df709 to 0x5c38f1 (0x41be17 bytes) 000065[1] 1748980361.374317 read 0x0269f6 thru 0x02cf22 (0x652d bytes) 000066[1] 1748980361.388829 write 0x85389b thru 0x85508e (0x17f4 bytes) 000067[1] 1748980361.395447 write 0x4ad9ad thru 0x4aefc3 (0x1617 bytes) 000068[0] 1748980361.401345 write 0x0b9693 thru 0x0c40ca (0xaa38 bytes) 000069[0] 1748980361.413522 mapread 0x4fd7bd thru 0x50001b (0x285f bytes) 000070[0] 1748980361.418299 read 0x77f6bb thru 0x781efc (0x2842 bytes) 000074[0] 1748980361.426654 write 0x342637 thru 0x347c4a (0x5614 bytes) 000076[0] 1748980361.432565 write 0x597924 thru 0x5a0e6a (0x9547 bytes) 000077[0] 1748980361.438849 mapwrite 0x689bfd thru 0x6937d8 (0x9bdc bytes) 000078[0] 1748980361.463835 write 0x215004 thru 0x215484 (0x481 bytes) 000080[1] 1748980361.472426 trunc from 0x85508f to 0x81ad6d (0x3a321 bytes) 000082[1] 1748980361.491546 mapwrite 0x915f93 thru 0x91a7de (0x484c bytes) 000084[0] 1748980361.534263 mapread 0x07300a thru 0x07817a (0x5171 bytes) 000085[1] 1748980361.557487 mapwrite 0x8141fe thru 0x81b95e (0x7761 bytes) 000086[1] 1748980361.586463 read 0x7feab0 thru 0x809f67 (0xb4b8 bytes) 000087[1] 1748980361.602729 write 0x9befd5 thru 0x9cc231 (0xd25d bytes) 000093[0] 1748980361.612695 read 0x7599a6 thru 0x76966c (0xfcc7 bytes) 000097[1] 1748980361.635689 write 0x49fe07 thru 0x4a3a55 (0x3c4f bytes) 000098[0] 1748980361.641883 trunc from 0x9cc232 to 0x380bd7 (0x64b65a bytes) 000099[1] 1748980361.656661 trunc from 0x380bd8 to 0x626069 (0x2a5492 bytes) 000100[0] 1748980361.663209 write 0x52d160 thru 0x534d5c (0x7bfd bytes) 000102[0] 1748980361.675282 trunc from 0x62606a to 0x76dd3a (0x147cd1 bytes) 000103[1] 1748980361.680612 trunc from 0x76dd3b to 0x6afe52 (0xbdee8 bytes) 000105[1] 1748980361.697599 read 0x428f4e thru 0x43192f (0x89e2 bytes) 000107[1] 1748980361.716042 mapwrite 0x7e041f thru 0x7ee2ac (0xde8e bytes) 000108[1] 1748980361.744681 trunc from 0x7ee2ad to 0x2e3724 (0x50ab88 bytes) 000110[1] 1748980361.756447 write 0x9fcc3a thru 0x9fffff (0x33c6 bytes) 000111[1] 1748980361.761672 mapwrite 0x44e6e1 thru 0x45c764 (0xe084 bytes) 000112[1] 1748980361.803514 write 0x6f4816 thru 0x6f934c (0x4b37 bytes) 000115[1] 1748980361.809395 write 0x489d73 thru 0x491977 (0x7c05 bytes) 000116[0] 1748980361.811960 mapwrite 0x3690d3 thru 0x3750de (0xc00c bytes) 000119[0] 1748980361.856120 trunc from 0xa00000 to 0x86733f (0x198cc0 bytes) 000120[0] 1748980361.868036 mapread 0x3dd5dc thru 0x3eb89e (0xe2c3 bytes) 000122[1] 1748980361.889740 write 0x6fbbe8 thru 0x70b12d (0xf546 bytes) 000123[1] 1748980361.896730 mapread 0x3a8534 thru 0x3a99e4 (0x14b1 bytes) 000124[1] 1748980361.899242 mapwrite 0x915ac6 thru 0x91f462 (0x999d bytes) 000126[0] 1748980361.933882 mapwrite 0x95e933 thru 0x96d006 (0xe6d4 bytes) 000129[1] 1748980361.967003 mapwrite 0x13fc68 thru 0x14178f (0x1b28 bytes) 000132[1] 1748980361.997972 trunc from 0x96d007 to 0x30ebc4 (0x65e442 bytes) 000133[1] 1748980362.005696 trunc from 0x30ebc5 to 0x1a7f3e (0x166c86 bytes) 000136[0] 1748980362.013821 mapwrite 0x1eb48d thru 0x1f6cb4 (0xb828 bytes) 000138[1] 1748980362.047799 trunc from 0x1f6cb5 to 0x3e414d (0x1ed499 bytes) 000139[0] 1748980362.056232 mapwrite 0x1e2fe2 thru 0x1ec823 (0x9842 bytes) 000140[1] 1748980362.091035 write 0x7d8ad9 thru 0x7e6981 (0xdea9 bytes) 000141[1] 1748980362.097686 mapread 0x68e248 thru 0x691a70 (0x3829 bytes) 000142[1] 1748980362.102255 read 0x5d81c0 thru 0x5db4d2 (0x3313 bytes) 000143[1] 1748980362.110718 mapwrite 0x4bd206 thru 0x4c768d (0xa488 bytes) 000145[1] 1748980362.137960 read 0x76337f thru 0x76422f (0xeb1 bytes) 000146[1] 1748980362.138288 trunc from 0x7e6982 to 0x828702 (0x41d81 bytes) 000148[0] 1748980362.147256 trunc from 0x828703 to 0x4a6897 (0x381e6b bytes) 000150[0] 1748980362.166147 mapread 0x080740 thru 0x09042a (0xfceb bytes) 000153[1] 1748980362.189089 mapwrite 0x94b8ac thru 0x94c23e (0x993 bytes) 000154[1] 1748980362.219714 read 0x5d992a thru 0x5e3604 (0x9cdb bytes) 000155[1] 1748980362.229222 write 0x4c18f7 thru 0x4ce30d (0xca17 bytes) 000156[0] 1748980362.235544 mapread 0x06aadc thru 0x06e9b5 (0x3eda bytes) 000159[0] 1748980362.259631 write 0x85b37d thru 0x862fa4 (0x7c28 bytes) 000160[0] 1748980362.267370 read 0x473977 thru 0x47f9c7 (0xc051 bytes) 000161[0] 1748980362.275625 write 0x3f942d thru 0x403b51 (0xa725 bytes) 000164[1] 1748980362.310846 mapwrite 0x498637 thru 0x49c0bb (0x3a85 bytes) 000166[0] 1748980362.346300 trunc from 0x94c23f to 0x62befc (0x320342 bytes) 000167[0] 1748980362.354948 mapread 0x020c57 thru 0x021aad (0xe57 bytes) 000168[1] 1748980362.357853 trunc from 0x62befd to 0x7319f4 (0x105af8 bytes) 000169[1] 1748980362.367669 write 0x67efa8 thru 0x68b66a (0xc6c3 bytes) 000170[0] 1748980362.374769 mapread 0x49b5c7 thru 0x49d898 (0x22d2 bytes) 000171[1] 1748980362.386485 read 0x024f53 thru 0x03341a (0xe4c8 bytes) 000172[1] 1748980362.389742 read 0x310047 thru 0x31653a (0x64f4 bytes) truncating to largest ever: 0x9f883e 000173[1] 1748980362.406574 trunc from 0x7319f5 to 0x9f883d (0x2c6e49 bytes) 000174[0] 1748980362.415310 mapwrite 0x6a7956 thru 0x6b689a (0xef45 bytes) 000175[1] 1748980362.452624 read 0x89fb07 thru 0x8a0fec (0x14e6 bytes) 000176[0] 1748980362.465862 trunc from 0x9f883e to 0x6a939e (0x34f49f bytes) 000177[0] 1748980362.476513 mapread 0x25f5df thru 0x26d4bd (0xdedf bytes) 000179[1] 1748980362.500188 read 0x37625d thru 0x37bce8 (0x5a8c bytes) 000180[1] 1748980362.510721 trunc from 0x6a939f to 0x715623 (0x6c285 bytes) 000182[0] 1748980362.518643 trunc from 0x715624 to 0x022d9e (0x6f2885 bytes) 000185[1] 1748980362.526147 read 0x00033d thru 0x00af22 (0xabe6 bytes) 000186[0] 1748980362.532158 mapwrite 0x882480 thru 0x88ba7a (0x95fb bytes) 000187[1] 1748980362.582316 trunc from 0x88ba7b to 0x1fc29b (0x68f7df bytes) 000189[0] 1748980362.605339 trunc from 0x1fc29c to 0x3e2cf8 (0x1e6a5d bytes) 000190[0] 1748980362.617154 mapread 0x268aa9 thru 0x272723 (0x9c7b bytes) 000191[1] 1748980362.636318 write 0x45db2a thru 0x46a1f2 (0xc6c9 bytes) 000192[1] 1748980362.647289 mapwrite 0x4cfa47 thru 0x4d30dd (0x3697 bytes) 000193[1] 1748980362.702705 read 0x00593d thru 0x00fe0a (0xa4ce bytes) 000194[0] 1748980362.707257 write 0x4848c9 thru 0x488c2e (0x4366 bytes) 000196[0] 1748980362.725526 read 0x0f577d thru 0x103a56 (0xe2da bytes) 000199[0] 1748980362.756009 write 0x789f42 thru 0x798367 (0xe426 bytes) 000200[0] 1748980362.764016 mapwrite 0x3d91cd thru 0x3db3f7 (0x222b bytes) 000203[1] 1748980362.794425 mapwrite 0x3c740a thru 0x3c8819 (0x1410 bytes) 000207[0] 1748980362.825900 trunc from 0x798368 to 0x5ca4bf (0x1cdea8 bytes) 000209[1] 1748980362.839259 mapwrite 0x86d37f thru 0x86e8a5 (0x1527 bytes) 000210[1] 1748980362.874210 trunc from 0x86e8a6 to 0x33bc84 (0x532c21 bytes) 000212[1] 1748980362.885275 mapread 0x0d79a5 thru 0x0e2f1a (0xb576 bytes) 000215[1] 1748980362.901390 mapwrite 0x552e79 thru 0x55bc39 (0x8dc1 bytes) 000217[1] 1748980362.931709 mapread 0x068db9 thru 0x0774f7 (0xe73f bytes) 000218[0] 1748980362.951468 mapwrite 0x2ac5b8 thru 0x2b079e (0x41e7 bytes) 000219[0] 1748980362.975240 mapwrite 0x32a54c thru 0x33076b (0x6220 bytes) 000220[1] 1748980362.999531 read 0x0b6896 thru 0x0bc06f (0x57da bytes) 000222[1] 1748980363.012197 mapwrite 0x23dfba thru 0x241156 (0x319d bytes) 000223[0] 1748980363.035970 trunc from 0x55bc3a to 0x0df40e (0x47c82b bytes) 000226[0] 1748980363.048822 trunc from 0x0df40f to 0x76b919 (0x68c50b bytes) 000227[1] 1748980363.053891 write 0x825ced thru 0x82851f (0x2833 bytes) 000228[0] 1748980363.059114 mapread 0x810aeb thru 0x81ac50 (0xa166 bytes) 000232[1] 1748980363.074564 read 0x6e75dc thru 0x6ec29e (0x4cc3 bytes) 000233[1] 1748980363.077020 read 0x8225d7 thru 0x8281ac (0x5bd6 bytes) 000234[1] 1748980363.078999 read 0x081795 thru 0x08857b (0x6de7 bytes) 000236[1] 1748980363.087651 mapread 0x15dd8e thru 0x167c3a (0x9ead bytes) 000237[1] 1748980363.118842 write 0x45adea thru 0x466fe2 (0xc1f9 bytes) 000238[0] 1748980363.130390 read 0x7587f6 thru 0x75d235 (0x4a40 bytes) 000241[0] 1748980363.140144 read 0x7bf3e0 thru 0x7c68bb (0x74dc bytes) 000243[1] 1748980363.155387 write 0x87a8f5 thru 0x886ed6 (0xc5e2 bytes) 000245[1] 1748980363.159944 read 0x61638c thru 0x619eb2 (0x3b27 bytes) 000248[0] 1748980363.172188 mapread 0x77a88f thru 0x77d34a (0x2abc bytes) 000250[1] 1748980363.189457 trunc from 0x886ed7 to 0x332cb9 (0x55421d bytes) 000252[0] 1748980363.206450 read 0x1b9c58 thru 0x1bfe0a (0x61b3 bytes) 000254[1] 1748980363.219434 read 0x0d4ba6 thru 0x0dd30b (0x8766 bytes) 000255[1] 1748980363.225815 read 0x32da52 thru 0x332cb9 (0x5268 bytes) 000258[1] 1748980363.238706 trunc from 0x332cba to 0x13ca16 (0x1f62a3 bytes) 000259[1] 1748980363.244099 mapread 0x08a092 thru 0x08f811 (0x5780 bytes) 000262[0] 1748980363.260771 read 0x139bb9 thru 0x13ca16 (0x2e5e bytes) 000263[0] 1748980363.266950 mapwrite 0x46fbc6 thru 0x47cf4c (0xd387 bytes) 000266[1] 1748980363.306781 trunc from 0x47cf4d to 0x14c482 (0x330aca bytes) 000267[0] 1748980363.317098 trunc from 0x14c483 to 0x105655 (0x46e2d bytes) 000268[1] 1748980363.327522 mapwrite 0x3358ce thru 0x338527 (0x2c5a bytes) 000270[0] 1748980363.351300 read 0x084d87 thru 0x08e6c6 (0x9940 bytes) 000271[0] 1748980363.361449 mapread 0x32e1a8 thru 0x338527 (0xa380 bytes) 000272[1] 1748980363.375087 trunc from 0x338528 to 0x3e497d (0xac456 bytes) 000273[0] 1748980363.384755 mapread 0x045b8b thru 0x0526bb (0xcb31 bytes) 000274[1] 1748980363.404784 read 0x00dcc7 thru 0x0182f2 (0xa62c bytes) 000275[0] 1748980363.422172 mapread 0x161caa thru 0x16b65c (0x99b3 bytes) 000276[1] 1748980363.437664 read 0x147739 thru 0x149e48 (0x2710 bytes) 000277[0] 1748980363.448007 mapread 0x227ba3 thru 0x22cffe (0x545c bytes) 000278[0] 1748980363.454504 mapread 0x02e97b thru 0x037dfd (0x9483 bytes) 000279[1] 1748980363.464800 trunc from 0x3e497e to 0x7ae331 (0x3c99b4 bytes) 000281[1] 1748980363.480485 mapwrite 0x236436 thru 0x23cf80 (0x6b4b bytes) 000283[0] 1748980363.520483 mapread 0x3d634b thru 0x3dda15 (0x76cb bytes) 000284[1] 1748980363.538113 read 0x4e8bb8 thru 0x4eff57 (0x73a0 bytes) 000287[1] 1748980363.549061 mapwrite 0x9dc4d9 thru 0x9e75c0 (0xb0e8 bytes) 000288[0] 1748980363.584591 trunc from 0x9e75c1 to 0x9cedf1 (0x187cf bytes) 000290[1] 1748980363.592832 write 0x607d00 thru 0x616e2f (0xf130 bytes) 000291[0] 1748980363.601160 read 0x2a9659 thru 0x2b82b5 (0xec5d bytes) 000292[1] 1748980363.625786 read 0x4ce1f5 thru 0x4d2887 (0x4693 bytes) 000294[1] 1748980363.638568 write 0x0fab48 thru 0x103e30 (0x92e9 bytes) 000295[0] 1748980363.671060 write 0x3e59bf thru 0x3e6b38 (0x117a bytes) 000301[0] 1748980363.698524 mapwrite 0x1ff79f thru 0x208c10 (0x9472 bytes) 000302[1] 1748980363.731869 read 0x45ac89 thru 0x45b777 (0xaef bytes) 000303[0] 1748980363.743154 write 0x8755ea thru 0x87a158 (0x4b6f bytes) 000305[0] 1748980363.751074 write 0x12b91c thru 0x136041 (0xa726 bytes) 000308[0] 1748980363.757283 mapread 0x484591 thru 0x48c049 (0x7ab9 bytes) 000309[0] 1748980363.766713 mapwrite 0x507d1c thru 0x514629 (0xc90e bytes) 000310[0] 1748980363.793961 mapwrite 0x976373 thru 0x977e0d (0x1a9b bytes) 000311[1] 1748980363.808120 trunc from 0x9cedf2 to 0x1846f0 (0x84a701 bytes) 000312[0] 1748980363.817925 read 0x0be0f0 thru 0x0caf24 (0xce35 bytes) 000314[0] 1748980363.831063 mapread 0x09b4c5 thru 0x09c848 (0x1384 bytes) 000317[1] 1748980363.842722 mapread 0x05e8f5 thru 0x06e18f (0xf89b bytes) 000318[1] 1748980363.864068 write 0x339c3e thru 0x34671a (0xcadd bytes) 000319[0] 1748980363.865552 mapread 0x2888d8 thru 0x2956a6 (0xcdcf bytes) 000323[0] 1748980363.914162 trunc from 0x34671b to 0x94c398 (0x605c7e bytes) 000324[0] 1748980363.927736 mapwrite 0x720202 thru 0x723b0c (0x390b bytes) 000326[0] 1748980363.970619 read 0x19f40e thru 0x1a2d8a (0x397d bytes) 000327[1] 1748980363.975164 read 0x731202 thru 0x73d26a (0xc069 bytes) 000329[0] 1748980363.988435 mapread 0x85cfb3 thru 0x860798 (0x37e6 bytes) 000330[1] 1748980363.992323 trunc from 0x94c399 to 0x4f8064 (0x454334 bytes) 000331[0] 1748980364.004601 mapwrite 0x0713e5 thru 0x076f6d (0x5b89 bytes) 000334[1] 1748980364.038898 read 0x44d4fe thru 0x450286 (0x2d89 bytes) 000336[1] 1748980364.048625 write 0x1416cc thru 0x14f6b0 (0xdfe5 bytes) 000337[0] 1748980364.054623 write 0x1f415a thru 0x1f4827 (0x6ce bytes) 000338[1] 1748980364.063107 read 0x40fdc0 thru 0x41aea3 (0xb0e4 bytes) 000340[0] 1748980364.070101 mapread 0x3b133c thru 0x3b8865 (0x752a bytes) 000341[0] 1748980364.082596 write 0x5b2215 thru 0x5b7a4d (0x5839 bytes) 000342[1] 1748980364.086901 trunc from 0x5b7a4e to 0x711d65 (0x15a318 bytes) 000343[0] 1748980364.100852 mapread 0x19b37c thru 0x1aa55f (0xf1e4 bytes) 000344[0] 1748980364.124440 mapread 0x3e470e thru 0x3e4da0 (0x693 bytes) 000345[1] 1748980364.130923 write 0x52c4af thru 0x52ffaf (0x3b01 bytes) 000346[0] 1748980364.146983 trunc from 0x711d66 to 0x23e130 (0x4d3c35 bytes) 000348[0] 1748980364.163768 mapread 0x1e482b thru 0x1e5ec3 (0x1699 bytes) 000349[0] 1748980364.169818 read 0x028cf2 thru 0x029d96 (0x10a5 bytes) 000353[1] 1748980364.189770 read 0x0acc65 thru 0x0b38a3 (0x6c3f bytes) 000356[0] 1748980364.205983 mapwrite 0x7d71a7 thru 0x7e65fd (0xf457 bytes) 000357[1] 1748980364.243318 trunc from 0x7e65fe to 0x63ec80 (0x1a797d bytes) 000360[0] 1748980364.259288 trunc from 0x63ec81 to 0x816de6 (0x1d8166 bytes) 000361[1] 1748980364.266659 mapread 0x7f6905 thru 0x7f9e95 (0x3591 bytes) 000363[1] 1748980364.287198 mapwrite 0x93148b thru 0x93c488 (0xaffe bytes) 000364[1] 1748980364.327141 read 0x410c0a thru 0x412766 (0x1b5d bytes) 000365[0] 1748980364.330871 mapwrite 0x46d1a1 thru 0x4797d7 (0xc637 bytes) 000366[0] 1748980364.368384 trunc from 0x93c489 to 0x6dfbf4 (0x25c894 bytes) 000369[0] 1748980364.385497 mapread 0x2f1420 thru 0x2fafc6 (0x9ba7 bytes) 000371[0] 1748980364.399803 write 0x35d63f thru 0x35fd07 (0x26c9 bytes) 000372[1] 1748980364.406040 read 0x6a7132 thru 0x6b2f90 (0xbe5f bytes) 000377[1] 1748980364.420711 mapread 0x0719b8 thru 0x075843 (0x3e8c bytes) 000378[0] 1748980364.428871 mapread 0x58a504 thru 0x5906ea (0x61e7 bytes) 000379[0] 1748980364.441473 mapwrite 0x13c4de thru 0x144e83 (0x89a6 bytes) 000381[1] 1748980364.472529 read 0x14fd94 thru 0x1534de (0x374b bytes) 000382[1] 1748980364.492888 read 0x1a58da thru 0x1b3878 (0xdf9f bytes) 000384[1] 1748980364.493247 read 0x4557f4 thru 0x459a4c (0x4259 bytes) 000385[0] 1748980364.494953 mapwrite 0x1cee9e thru 0x1dbfc7 (0xd12a bytes) 000386[0] 1748980364.527938 read 0x1a548b thru 0x1b27e3 (0xd359 bytes) 000387[0] 1748980364.542907 trunc from 0x6dfbf5 to 0x0023eb (0x6dd809 bytes) 000389[1] 1748980364.552932 write 0x892577 thru 0x895d5c (0x37e6 bytes) 000391[0] 1748980364.564067 trunc from 0x895d5d to 0x11aa7a (0x77b2e2 bytes) 000392[1] 1748980364.573669 trunc from 0x11aa7b to 0x1a48b7 (0x89e3d bytes) 000394[0] 1748980364.587193 write 0x0bf1f8 thru 0x0c02f3 (0x10fc bytes) 000395[0] 1748980364.594290 write 0x5dae14 thru 0x5e2518 (0x7705 bytes) 000396[0] 1748980364.597179 trunc from 0x5e2519 to 0x702a4e (0x120536 bytes) 000397[0] 1748980364.603740 mapread 0x1fb5ad thru 0x1fd35c (0x1db0 bytes) 000398[1] 1748980364.608641 trunc from 0x702a4f to 0x0e514f (0x61d8ff bytes) 000402[1] 1748980364.622810 mapread 0x0dbf86 thru 0x0e514f (0x91ca bytes) 000404[1] 1748980364.635600 write 0x7fcd46 thru 0x802975 (0x5c30 bytes) 000405[1] 1748980364.641060 write 0x96ff57 thru 0x97e411 (0xe4bb bytes) 000406[0] 1748980364.645544 trunc from 0x97e412 to 0x1eee87 (0x78f58a bytes) 000407[1] 1748980364.656626 read 0x0f26cb thru 0x0fc532 (0x9e68 bytes) 000409[1] 1748980364.669386 read 0x1b7416 thru 0x1bf8ff (0x84ea bytes) 000412[1] 1748980364.677453 mapwrite 0x7e8d3a thru 0x7f8b50 (0xfe17 bytes) 000414[0] 1748980364.721607 mapread 0x164f53 thru 0x16b180 (0x622e bytes) 000416[0] 1748980364.750419 read 0x485c08 thru 0x491f55 (0xc34e bytes) 000417[0] 1748980364.767059 read 0x7bdbc6 thru 0x7be75f (0xb9a bytes) 000420[1] 1748980364.768514 mapwrite 0x11649b thru 0x119acd (0x3633 bytes) 000422[0] 1748980364.789436 read 0x55143b thru 0x55cf9d (0xbb63 bytes) 000424[1] 1748980364.801948 trunc from 0x7f8b51 to 0x04f6dd (0x7a9473 bytes) 000425[0] 1748980364.815493 mapwrite 0x3e139d thru 0x3e6474 (0x50d8 bytes) 000426[1] 1748980364.843288 read 0x094663 thru 0x09a7cd (0x616b bytes) 000427[0] 1748980364.853621 write 0x25c69f thru 0x25fb33 (0x3495 bytes) 000429[1] 1748980364.864663 read 0x3779d7 thru 0x377ad2 (0xfc bytes) 000434[1] 1748980364.881249 mapwrite 0x4eea6c thru 0x4f9661 (0xabf6 bytes) 000435[1] 1748980364.910608 mapwrite 0x3e515f thru 0x3f32b4 (0xe156 bytes) 000436[0] 1748980364.924098 mapwrite 0x6a451e thru 0x6b1bf2 (0xd6d5 bytes) 000437[0] 1748980364.962715 read 0x4bfe06 thru 0x4cd035 (0xd230 bytes) 000438[0] 1748980364.973193 trunc from 0x6b1bf3 to 0x32921e (0x3889d4 bytes) 000442[0] 1748980364.987379 read 0x0bd976 thru 0x0c1e03 (0x448e bytes) 000447[1] 1748980365.005292 mapwrite 0x150aa3 thru 0x1600bb (0xf619 bytes) 000450[0] 1748980365.051891 mapread 0x1bdc0f thru 0x1cd338 (0xf72a bytes) 000451[0] 1748980365.076865 mapread 0x287a4e thru 0x291bce (0xa181 bytes) 000452[1] 1748980365.077200 trunc from 0x32921f to 0x5157ef (0x1ec5d1 bytes) 000453[0] 1748980365.089598 read 0x450b02 thru 0x4532d6 (0x27d5 bytes) 000455[0] 1748980365.104412 mapread 0x1e4a34 thru 0x1ea72a (0x5cf7 bytes) 000456[0] 1748980365.118618 read 0x056c65 thru 0x060c0b (0x9fa7 bytes) 000458[1] 1748980365.131457 mapread 0x2c9248 thru 0x2d7eaa (0xec63 bytes) 000459[0] 1748980365.147260 write 0x485a57 thru 0x4946bb (0xec65 bytes) 000460[0] 1748980365.157950 read 0x1fa4a3 thru 0x2031a7 (0x8d05 bytes) 000461[0] 1748980365.172540 read 0x29a35b thru 0x2a0e6f (0x6b15 bytes) 000462[1] 1748980365.172777 mapread 0x17ef55 thru 0x189469 (0xa515 bytes) 000463[0] 1748980365.190969 read 0x1236a6 thru 0x12b9e7 (0x8342 bytes) 000466[1] 1748980365.191488 read 0x35f82f thru 0x36446c (0x4c3e bytes) 000467[1] 1748980365.197244 write 0x9b2ee7 thru 0x9c0a7b (0xdb95 bytes) 000468[0] 1748980365.205297 mapread 0x7fd446 thru 0x802867 (0x5422 bytes) 000469[0] 1748980365.216929 trunc from 0x9c0a7c to 0x9a81c4 (0x188b7 bytes) 000470[0] 1748980365.225661 write 0x1e1ab0 thru 0x1e4617 (0x2b68 bytes) 000471[1] 1748980365.231342 mapwrite 0x822cde thru 0x829361 (0x6684 bytes) 000476[1] 1748980365.286258 mapwrite 0x5a8fe9 thru 0x5b4dfe (0xbe16 bytes) 000477[0] 1748980365.324297 mapread 0x10b22f thru 0x113330 (0x8102 bytes) 000483[0] 1748980365.345483 mapread 0x6dd79b thru 0x6dee87 (0x16ed bytes) 000485[0] 1748980365.355363 mapread 0x3b1f2f thru 0x3bdd79 (0xbe4b bytes) 000486[1] 1748980365.369386 read 0x71d45f thru 0x71f74f (0x22f1 bytes) 000487[0] 1748980365.378532 trunc from 0x9a81c5 to 0x8e5991 (0xc2833 bytes) 000488[1] 1748980365.390277 read 0x68e125 thru 0x68f128 (0x1004 bytes) 000490[0] 1748980365.398675 mapwrite 0x6adcef thru 0x6b5922 (0x7c34 bytes) 000491[0] 1748980365.426054 mapread 0x6a5554 thru 0x6abf59 (0x6a06 bytes) 000492[0] 1748980365.437665 write 0x2cd27c thru 0x2d8499 (0xb21e bytes) 000493[1] 1748980365.440551 mapwrite 0x135885 thru 0x138910 (0x308c bytes) 000498[1] 1748980365.486111 read 0x74dc51 thru 0x757039 (0x93e9 bytes) 000499[1] 1748980365.490319 mapwrite 0x319155 thru 0x31b73b (0x25e7 bytes) 000500[0] 1748980365.511834 write 0x3c3faa thru 0x3d31de (0xf235 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 365 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read 000004[0] 1748980365.725345 write_OD 0x570000 thru 0x57afff (0xb000 bytes) 000005[1] 1748980365.738289 write_OD 0x70b000 thru 0x718fff (0xe000 bytes) truncating to largest ever: 0x3a3b17 000006[1] 1748980365.750428 trunc from 0x719000 to 0x3a3b16 (0x3754e9 bytes) truncating to largest ever: 0x7f3ec7 000009[0] 1748980365.764196 trunc from 0x3a3b17 to 0x7f3ec6 (0x4503b0 bytes) 000013[1] 1748980365.772941 trunc from 0x7f3ec7 to 0x4693ff (0x38aac7 bytes) 000014[0] 1748980365.780908 write_OD 0x0f3000 thru 0x0fafff (0x8000 bytes) 000015[0] 1748980365.793554 write_OD 0x4e5000 thru 0x4f1fff (0xd000 bytes) 000016[0] 1748980365.805371 mapwrite 0x044000 thru 0x04929b (0x529c bytes) 000017[1] 1748980365.834206 mapread 0x139000 thru 0x146b57 (0xdb58 bytes) 000018[1] 1748980365.844350 write_OD 0x99b000 thru 0x9a3fff (0x9000 bytes) 000019[1] 1748980365.858647 read_OD 0x2e6000 thru 0x2f1fff (0xc000 bytes) 000020[0] 1748980365.861784 mapread 0x78a000 thru 0x78e71b (0x471c bytes) 000022[1] 1748980365.866118 read_OD 0x973000 thru 0x974fff (0x2000 bytes) 000023[1] 1748980365.867517 trunc from 0x9a4000 to 0x367d4c (0x63c2b3 bytes) 000025[0] 1748980365.878578 mapread 0x0ed000 thru 0x0f2e14 (0x5e15 bytes) 000027[1] 1748980365.887665 trunc from 0x367d4d to 0x4cde47 (0x1660fb bytes) 000028[1] 1748980365.893524 mapread 0x040000 thru 0x045578 (0x5579 bytes) 000030[0] 1748980365.900103 mapwrite 0x2b9000 thru 0x2bde59 (0x4e5a bytes) truncating to largest ever: 0x8d3771 000032[1] 1748980365.927540 trunc from 0x4cde48 to 0x8d3770 (0x405929 bytes) 000034[1] 1748980365.939224 mapwrite 0x4fc000 thru 0x507ca6 (0xbca7 bytes) 000038[1] 1748980365.970316 mapread 0x3be000 thru 0x3be4a8 (0x4a9 bytes) 000039[1] 1748980365.975056 trunc from 0x8d3771 to 0x475bbc (0x45dbb4 bytes) 000041[0] 1748980365.984199 mapread 0x031000 thru 0x035d68 (0x4d69 bytes) 000042[0] 1748980365.993470 read_OD 0x0e3000 thru 0x0e8fff (0x6000 bytes) 000043[0] 1748980365.995616 write_OD 0x218000 thru 0x223fff (0xc000 bytes) 000044[1] 1748980366.008758 trunc from 0x475bbd to 0x013d4d (0x461e6f bytes) 000047[1] 1748980366.020248 write_OD 0x77e000 thru 0x783fff (0x6000 bytes) 000049[0] 1748980366.035583 trunc from 0x784000 to 0x83322c (0xaf22d bytes) 000050[1] 1748980366.046376 trunc from 0x83322d to 0x6df9c3 (0x153869 bytes) 000052[1] 1748980366.053060 trunc from 0x6df9c4 to 0x21adb0 (0x4c4c13 bytes) 000056[1] 1748980366.066824 read_OD 0x05c000 thru 0x067fff (0xc000 bytes) 000057[1] 1748980366.072146 trunc from 0x21adb1 to 0x45b2a4 (0x2404f4 bytes) 000058[0] 1748980366.083162 write_OD 0x46f000 thru 0x476fff (0x8000 bytes) 000059[1] 1748980366.097133 mapread 0x46d000 thru 0x47542e (0x842f bytes) 000061[0] 1748980366.115989 mapwrite 0x4c7000 thru 0x4c8e0d (0x1e0e bytes) 000067[1] 1748980366.147832 write_OD 0x93c000 thru 0x93efff (0x3000 bytes) 000068[1] 1748980366.157898 trunc from 0x93f000 to 0x6fa9b9 (0x244646 bytes) 000069[1] 1748980366.162914 mapread 0x277000 thru 0x279f30 (0x2f31 bytes) 000071[0] 1748980366.167230 read_OD 0x54f000 thru 0x559fff (0xb000 bytes) 000074[0] 1748980366.173739 write_OD 0x299000 thru 0x29cfff (0x4000 bytes) 000075[1] 1748980366.183828 mapwrite 0x5a7000 thru 0x5a7a3e (0xa3f bytes) 000076[0] 1748980366.197079 read_OD 0x341000 thru 0x34afff (0xa000 bytes) 000077[1] 1748980366.202441 read_OD 0x336000 thru 0x33dfff (0x8000 bytes) 000078[0] 1748980366.204484 write_OD 0x3cb000 thru 0x3d2fff (0x8000 bytes) 000079[1] 1748980366.215829 trunc from 0x6fa9ba to 0x172215 (0x5887a4 bytes) 000080[0] 1748980366.224742 mapread 0x0ec000 thru 0x0fa9e7 (0xe9e8 bytes) 000084[1] 1748980366.245908 write_OD 0x99a000 thru 0x9a1fff (0x8000 bytes) 000085[0] 1748980366.260027 write_OD 0x90a000 thru 0x918fff (0xf000 bytes) 000087[1] 1748980366.272443 mapread 0x59a000 thru 0x59ab6e (0xb6f bytes) 000088[0] 1748980366.273922 mapwrite 0x5bb000 thru 0x5bd67c (0x267d bytes) 000089[0] 1748980366.311810 mapwrite 0x7d7000 thru 0x7e4ee3 (0xdee4 bytes) 000090[0] 1748980366.325699 write_OD 0x0e4000 thru 0x0eefff (0xb000 bytes) 000091[1] 1748980366.342197 write_OD 0x1ef000 thru 0x1f7fff (0x9000 bytes) 000098[0] 1748980366.355629 mapread 0x573000 thru 0x57d844 (0xa845 bytes) truncating to largest ever: 0x96dc2c 000099[1] 1748980366.367635 trunc from 0x9a2000 to 0x96dc2b (0x343d4 bytes) 000100[1] 1748980366.373926 mapwrite 0x4f9000 thru 0x4ff15f (0x6160 bytes) 000103[1] 1748980366.401558 read_OD 0x53a000 thru 0x540fff (0x7000 bytes) 000104[1] 1748980366.407779 mapread 0x621000 thru 0x62b3ee (0xa3ef bytes) 000106[1] 1748980366.431696 write_OD 0x051000 thru 0x058fff (0x8000 bytes) 000107[1] 1748980366.449514 read_OD 0x41d000 thru 0x41dfff (0x1000 bytes) 000108[0] 1748980366.451858 write_OD 0x39c000 thru 0x3a6fff (0xb000 bytes) 000110[1] 1748980366.466109 mapwrite 0x807000 thru 0x807d49 (0xd4a bytes) 000112[0] 1748980366.504377 mapread 0x785000 thru 0x789076 (0x4077 bytes) 000113[0] 1748980366.518948 write_OD 0x910000 thru 0x910fff (0x1000 bytes) 000114[0] 1748980366.527646 mapwrite 0x818000 thru 0x81b7f8 (0x37f9 bytes) 000115[1] 1748980366.543357 mapwrite 0x2eb000 thru 0x2f782a (0xc82b bytes) 000116[0] 1748980366.573553 trunc from 0x96dc2c to 0x841da1 (0x12be8a bytes) 000117[0] 1748980366.583121 read_OD 0x08f000 thru 0x09afff (0xc000 bytes) 000118[1] 1748980366.588148 read_OD 0x215000 thru 0x21afff (0x6000 bytes) 000123[0] 1748980366.591830 mapread 0x33a000 thru 0x33f1d8 (0x51d9 bytes) 000124[0] 1748980366.625264 write_OD 0x581000 thru 0x58dfff (0xd000 bytes) 000126[0] 1748980366.648132 mapread 0x135000 thru 0x13db06 (0x8b07 bytes) 000129[1] 1748980366.659004 mapwrite 0x6e0000 thru 0x6e78bc (0x78bd bytes) 000130[0] 1748980366.695761 mapwrite 0x5d9000 thru 0x5e3630 (0xa631 bytes) 000131[1] 1748980366.733006 trunc from 0x841da2 to 0x8a11a4 (0x5f403 bytes) 000132[1] 1748980366.739059 trunc from 0x8a11a5 to 0x7a75b1 (0xf9bf3 bytes) 000133[0] 1748980366.746172 write_OD 0x621000 thru 0x62dfff (0xd000 bytes) 000135[1] 1748980366.761043 mapwrite 0x953000 thru 0x954f06 (0x1f07 bytes) 000136[0] 1748980366.783011 trunc from 0x954f07 to 0x055baa (0x8ff35c bytes) 000138[0] 1748980366.793115 write_OD 0x43d000 thru 0x449fff (0xd000 bytes) 000142[1] 1748980366.809802 write_OD 0x744000 thru 0x74ffff (0xc000 bytes) 000143[0] 1748980366.822287 mapread 0x4e4000 thru 0x4f14a7 (0xd4a8 bytes) 000145[0] 1748980366.838704 mapwrite 0x742000 thru 0x74f95c (0xd95d bytes) 000148[1] 1748980366.876414 write_OD 0x2b3000 thru 0x2bffff (0xd000 bytes) 000149[0] 1748980366.892243 mapread 0x17d000 thru 0x18c351 (0xf352 bytes) 000152[0] 1748980366.910027 write_OD 0x9fe000 thru 0x9fefff (0x1000 bytes) 000153[0] 1748980366.924122 mapread 0x107000 thru 0x112f59 (0xbf5a bytes) 000157[1] 1748980366.938939 mapwrite 0x2d7000 thru 0x2dd650 (0x6651 bytes) 000158[1] 1748980366.971063 read_OD 0x01e000 thru 0x022fff (0x5000 bytes) 000159[0] 1748980366.975619 trunc from 0x9ff000 to 0x4d1ea9 (0x52d156 bytes) 000160[0] 1748980366.986117 read_OD 0x056000 thru 0x05efff (0x9000 bytes) 000162[0] 1748980366.991847 mapwrite 0x918000 thru 0x9218c4 (0x98c5 bytes) 000165[0] 1748980367.065247 read_OD 0x561000 thru 0x56bfff (0xb000 bytes) 000167[1] 1748980367.077313 write_OD 0x09e000 thru 0x09ffff (0x2000 bytes) 000168[0] 1748980367.091481 mapread 0x76c000 thru 0x76ff65 (0x3f66 bytes) 000169[0] 1748980367.110514 write_OD 0x28f000 thru 0x29afff (0xc000 bytes) 000171[1] 1748980367.128894 mapread 0x309000 thru 0x314ece (0xbecf bytes) 000172[0] 1748980367.174704 trunc from 0x9218c5 to 0x8747b2 (0xad112 bytes) 000175[1] 1748980367.195521 mapread 0x7a2000 thru 0x7a3461 (0x1462 bytes) 000178[1] 1748980367.203079 mapwrite 0x831000 thru 0x8396cd (0x86ce bytes) 000181[1] 1748980367.239870 read_OD 0x485000 thru 0x493fff (0xf000 bytes) 000186[0] 1748980367.246090 trunc from 0x8747b3 to 0x1cc969 (0x6a7e49 bytes) 000187[1] 1748980367.253103 read_OD 0x014000 thru 0x019fff (0x6000 bytes) 000188[1] 1748980367.260042 trunc from 0x1cc96a to 0x4cfe45 (0x3034dc bytes) 000190[1] 1748980367.272303 mapread 0x3e1000 thru 0x3e346c (0x246d bytes) 000196[0] 1748980367.297860 read_OD 0x240000 thru 0x24dfff (0xe000 bytes) 000197[0] 1748980367.309350 mapread 0x193000 thru 0x19e2c6 (0xb2c7 bytes) 000199[0] 1748980367.337316 write_OD 0x28d000 thru 0x29afff (0xe000 bytes) 000201[1] 1748980367.354884 mapread 0x015000 thru 0x016f04 (0x1f05 bytes) 000202[0] 1748980367.357416 mapread 0x283000 thru 0x28bb20 (0x8b21 bytes) 000204[1] 1748980367.367741 mapwrite 0x227000 thru 0x233abf (0xcac0 bytes) 000207[0] 1748980367.402982 write_OD 0x3ca000 thru 0x3cafff (0x1000 bytes) 000209[1] 1748980367.420182 mapread 0x279000 thru 0x2855ab (0xc5ac bytes) 000212[0] 1748980367.435261 trunc from 0x4cfe46 to 0x8992c8 (0x3c9483 bytes) 000213[1] 1748980367.446458 mapread 0x192000 thru 0x1998a9 (0x78aa bytes) 000215[1] 1748980367.460473 trunc from 0x8992c9 to 0x56d739 (0x32bb8f bytes) 000216[1] 1748980367.466217 write_OD 0x8f1000 thru 0x8fffff (0xf000 bytes) 000217[0] 1748980367.483593 read_OD 0x07a000 thru 0x087fff (0xe000 bytes) 000220[0] 1748980367.487183 trunc from 0x900000 to 0x0fb4ce (0x804b31 bytes) 000221[0] 1748980367.494052 mapwrite 0x386000 thru 0x392f7b (0xcf7c bytes) 000223[0] 1748980367.537941 mapread 0x1ad000 thru 0x1af3b0 (0x23b1 bytes) 000224[0] 1748980367.538213 write_OD 0x40f000 thru 0x41bfff (0xd000 bytes) 000225[1] 1748980367.555485 mapwrite 0x05e000 thru 0x061eec (0x3eed bytes) 000227[0] 1748980367.580532 mapread 0x242000 thru 0x24d11d (0xb11e bytes) 000231[1] 1748980367.600250 write_OD 0x9e7000 thru 0x9f2fff (0xc000 bytes) 000232[0] 1748980367.615466 mapwrite 0x2f3000 thru 0x2f4ebe (0x1ebf bytes) 000233[1] 1748980367.632435 mapread 0x48a000 thru 0x4924ff (0x8500 bytes) 000234[0] 1748980367.646588 mapread 0x66b000 thru 0x674cd8 (0x9cd9 bytes) 000235[1] 1748980367.657819 write_OD 0x787000 thru 0x792fff (0xc000 bytes) 000236[1] 1748980367.671146 trunc from 0x9f3000 to 0x7bae80 (0x23817f bytes) 000241[0] 1748980367.686516 read_OD 0x35e000 thru 0x367fff (0xa000 bytes) 000243[1] 1748980367.692675 read_OD 0x285000 thru 0x28bfff (0x7000 bytes) 000245[1] 1748980367.695733 read_OD 0x398000 thru 0x3a5fff (0xe000 bytes) 000246[1] 1748980367.697992 mapread 0x0a2000 thru 0x0ad23c (0xb23d bytes) 000248[1] 1748980367.734012 trunc from 0x7bae81 to 0x3d63cc (0x3e4ab4 bytes) 000250[0] 1748980367.751679 read_OD 0x115000 thru 0x122fff (0xe000 bytes) 000251[0] 1748980367.757715 mapread 0x214000 thru 0x222060 (0xe061 bytes) 000253[0] 1748980367.776972 mapwrite 0x1ad000 thru 0x1b1df7 (0x4df8 bytes) 000256[1] 1748980367.821094 trunc from 0x3d63cd to 0x36e295 (0x68137 bytes) 000257[1] 1748980367.828126 mapwrite 0x091000 thru 0x0990e5 (0x80e6 bytes) 000258[0] 1748980367.852871 mapwrite 0x24e000 thru 0x259e2f (0xbe30 bytes) 000261[0] 1748980367.895831 trunc from 0x36e296 to 0x4caad8 (0x15c843 bytes) 000262[1] 1748980367.904667 trunc from 0x4caad9 to 0x3e13e4 (0xe96f4 bytes) 000263[0] 1748980367.914517 write_OD 0x28e000 thru 0x28efff (0x1000 bytes) 000265[1] 1748980367.929974 trunc from 0x3e13e5 to 0x66079c (0x27f3b8 bytes) 000266[1] 1748980367.938238 read_OD 0x64d000 thru 0x655fff (0x9000 bytes) 000267[0] 1748980367.942555 mapwrite 0x909000 thru 0x9100b5 (0x70b6 bytes) 000268[0] 1748980367.985280 mapwrite 0x357000 thru 0x35f876 (0x8877 bytes) 000270[0] 1748980368.015811 mapwrite 0x781000 thru 0x7837f4 (0x27f5 bytes) 000271[0] 1748980368.028901 mapread 0x890000 thru 0x894b12 (0x4b13 bytes) 000273[0] 1748980368.032380 mapwrite 0x6c1000 thru 0x6d0b3e (0xfb3f bytes) 000274[1] 1748980368.045885 write_OD 0x992000 thru 0x994fff (0x3000 bytes) 000276[1] 1748980368.063572 mapwrite 0x584000 thru 0x5920f8 (0xe0f9 bytes) 000277[0] 1748980368.108715 read_OD 0x91b000 thru 0x922fff (0x8000 bytes) 000278[1] 1748980368.117634 mapwrite 0x350000 thru 0x352c17 (0x2c18 bytes) 000280[0] 1748980368.138026 write_OD 0x2c4000 thru 0x2c7fff (0x4000 bytes) 000281[0] 1748980368.149671 mapread 0x594000 thru 0x599fab (0x5fac bytes) 000283[0] 1748980368.161764 write_OD 0x2c1000 thru 0x2c9fff (0x9000 bytes) 000286[0] 1748980368.175923 write_OD 0x9d4000 thru 0x9d7fff (0x4000 bytes) 000289[1] 1748980368.188392 trunc from 0x9d8000 to 0x7c3500 (0x214aff bytes) 000290[0] 1748980368.197421 trunc from 0x7c3501 to 0x8622af (0x9edaf bytes) 000293[0] 1748980368.210046 read_OD 0x4f3000 thru 0x4f8fff (0x6000 bytes) 000295[0] 1748980368.213929 trunc from 0x8622b0 to 0x6af162 (0x1b314d bytes) 000296[1] 1748980368.223536 trunc from 0x6af163 to 0x93a22b (0x28b0c9 bytes) 000297[0] 1748980368.229272 mapwrite 0x624000 thru 0x62c080 (0x8081 bytes) 000299[1] 1748980368.275348 write_OD 0x383000 thru 0x38bfff (0x9000 bytes) 000300[0] 1748980368.291609 mapread 0x0c2000 thru 0x0ce585 (0xc586 bytes) 000301[0] 1748980368.306214 mapwrite 0x28b000 thru 0x28c10e (0x110f bytes) 000304[1] 1748980368.326866 trunc from 0x93a22c to 0x6e36ea (0x256b41 bytes) 000306[1] 1748980368.338207 trunc from 0x6e36eb to 0x668915 (0x7add5 bytes) 000307[0] 1748980368.342873 read_OD 0x088000 thru 0x08cfff (0x5000 bytes) 000308[1] 1748980368.348632 mapwrite 0x46b000 thru 0x479a6f (0xea70 bytes) 000309[0] 1748980368.386877 write_OD 0x528000 thru 0x534fff (0xd000 bytes) 000310[1] 1748980368.400284 trunc from 0x668916 to 0x42475e (0x2441b7 bytes) 000311[0] 1748980368.410274 trunc from 0x42475f to 0x1d5039 (0x24f725 bytes) 000312[1] 1748980368.415813 trunc from 0x1d503a to 0x89bfb7 (0x6c6f7e bytes) 000313[1] 1748980368.423516 trunc from 0x89bfb8 to 0x92d636 (0x9167f bytes) 000314[0] 1748980368.426916 read_OD 0x481000 thru 0x48bfff (0xb000 bytes) 000315[0] 1748980368.434056 read_OD 0x791000 thru 0x79efff (0xe000 bytes) 000319[0] 1748980368.437656 trunc from 0x92d637 to 0x130fde (0x7fc658 bytes) 000320[0] 1748980368.447587 trunc from 0x130fdf to 0x06ee5b (0xc2183 bytes) 000323[1] 1748980368.457003 trunc from 0x06ee5c to 0x6e88bf (0x679a64 bytes) 000324[0] 1748980368.464026 write_OD 0x6f2000 thru 0x6f5fff (0x4000 bytes) 000325[1] 1748980368.478301 trunc from 0x6f6000 to 0x19bffe (0x55a001 bytes) 000327[0] 1748980368.490700 trunc from 0x19bfff to 0x4cffe1 (0x333fe3 bytes) 000329[1] 1748980368.505572 read_OD 0x3f1000 thru 0x3fcfff (0xc000 bytes) 000331[1] 1748980368.513693 read_OD 0x274000 thru 0x274fff (0x1000 bytes) 000334[0] 1748980368.515042 write_OD 0x932000 thru 0x938fff (0x7000 bytes) 000335[0] 1748980368.528993 read_OD 0x3e0000 thru 0x3e4fff (0x5000 bytes) 000336[0] 1748980368.532045 mapwrite 0x754000 thru 0x75c9ee (0x89ef bytes) 000339[1] 1748980368.577759 write_OD 0x8c9000 thru 0x8d0fff (0x8000 bytes) 000341[1] 1748980368.594631 mapwrite 0x1b5000 thru 0x1b876a (0x376b bytes) 000342[0] 1748980368.640165 trunc from 0x939000 to 0x0722d2 (0x8c6d2d bytes) 000343[1] 1748980368.651571 mapwrite 0x765000 thru 0x7727d2 (0xd7d3 bytes) 000344[0] 1748980368.690085 write_OD 0x906000 thru 0x90ffff (0xa000 bytes) 000345[1] 1748980368.704427 mapwrite 0x7f8000 thru 0x80065d (0x865e bytes) 000346[0] 1748980368.735304 write_OD 0x60f000 thru 0x617fff (0x9000 bytes) 000347[1] 1748980368.750211 read_OD 0x393000 thru 0x39dfff (0xb000 bytes) 000348[0] 1748980368.753650 mapread 0x22f000 thru 0x239388 (0xa389 bytes) 000349[0] 1748980368.766032 trunc from 0x910000 to 0x241367 (0x6cec98 bytes) 000352[1] 1748980368.777278 trunc from 0x241368 to 0x17d502 (0xc3e65 bytes) 000358[0] 1748980368.796976 read_OD 0x07b000 thru 0x07bfff (0x1000 bytes) 000359[0] 1748980368.801668 mapwrite 0x3c2000 thru 0x3d1775 (0xf776 bytes) 000362[0] 1748980368.848505 write_OD 0x8a0000 thru 0x8a5fff (0x6000 bytes) 000363[0] 1748980368.865824 trunc from 0x8a6000 to 0x3ef43b (0x4b6bc4 bytes) 000364[0] 1748980368.875555 mapwrite 0x7dd000 thru 0x7e2b6c (0x5b6d bytes) 000365[1] 1748980368.900801 mapwrite 0x7a7000 thru 0x7b58f7 (0xe8f8 bytes) 000367[0] 1748980368.944717 mapread 0x7dc000 thru 0x7e2b14 (0x6b15 bytes) 000368[0] 1748980368.960096 mapread 0x360000 thru 0x36b0fa (0xb0fb bytes) 000370[1] 1748980368.972866 read_OD 0x0f4000 thru 0x0fbfff (0x8000 bytes) 000373[1] 1748980368.975246 trunc from 0x7e2b6d to 0x539b61 (0x2a900b bytes) 000374[1] 1748980368.989567 trunc from 0x539b62 to 0x5b1c27 (0x780c6 bytes) 000375[0] 1748980368.997381 read_OD 0x369000 thru 0x373fff (0xb000 bytes) 000376[0] 1748980369.003944 read_OD 0x22d000 thru 0x234fff (0x8000 bytes) 000377[0] 1748980369.006697 write_OD 0x06c000 thru 0x06efff (0x3000 bytes) 000384[0] 1748980369.022188 trunc from 0x5b1c28 to 0x245a8a (0x36c19d bytes) 000385[0] 1748980369.032613 mapread 0x16e000 thru 0x16fec2 (0x1ec3 bytes) 000386[0] 1748980369.037956 mapwrite 0x3b4000 thru 0x3c3387 (0xf388 bytes) 000387[0] 1748980369.069817 write_OD 0x8a2000 thru 0x8a3fff (0x2000 bytes) 000388[0] 1748980369.084558 read_OD 0x0aa000 thru 0x0b0fff (0x7000 bytes) 000389[1] 1748980369.089015 write_OD 0x5fe000 thru 0x600fff (0x3000 bytes) 000393[1] 1748980369.102948 mapread 0x50c000 thru 0x50c1bd (0x1be bytes) 000394[1] 1748980369.132016 trunc from 0x8a4000 to 0x6c1af1 (0x1e250e bytes) 000399[1] 1748980369.153265 mapread 0x4e3000 thru 0x4e9bf6 (0x6bf7 bytes) 000401[1] 1748980369.165549 read_OD 0x4b0000 thru 0x4bafff (0xb000 bytes) 000402[1] 1748980369.169589 mapread 0x567000 thru 0x56c8c6 (0x58c7 bytes) 000403[0] 1748980369.182528 write_OD 0x112000 thru 0x115fff (0x4000 bytes) 000405[1] 1748980369.196918 mapread 0x663000 thru 0x66740f (0x4410 bytes) 000407[1] 1748980369.203142 read_OD 0x10a000 thru 0x110fff (0x7000 bytes) 000409[0] 1748980369.205475 read_OD 0x53b000 thru 0x53bfff (0x1000 bytes) 000411[0] 1748980369.206855 mapread 0x493000 thru 0x496804 (0x3805 bytes) 000414[1] 1748980369.220234 write_OD 0x095000 thru 0x096fff (0x2000 bytes) 000416[1] 1748980369.235648 read_OD 0x121000 thru 0x127fff (0x7000 bytes) 000417[0] 1748980369.237812 read_OD 0x176000 thru 0x17efff (0x9000 bytes) 000418[1] 1748980369.239591 trunc from 0x6c1af2 to 0x63d5ec (0x84505 bytes) 000419[1] 1748980369.249334 mapread 0x06e000 thru 0x07d73b (0xf73c bytes) 000420[0] 1748980369.286162 mapwrite 0x824000 thru 0x82d883 (0x9884 bytes) 000421[0] 1748980369.340103 trunc from 0x82d884 to 0x915a84 (0xe8201 bytes) 000422[0] 1748980369.344574 mapwrite 0x8a3000 thru 0x8a33a4 (0x3a5 bytes) 000424[0] 1748980369.358355 trunc from 0x915a85 to 0x354ccb (0x5c0db9 bytes) 000425[1] 1748980369.366110 write_OD 0x0da000 thru 0x0e5fff (0xc000 bytes) 000426[0] 1748980369.382719 mapread 0x217000 thru 0x21e1a6 (0x71a7 bytes) 000429[1] 1748980369.393304 write_OD 0x502000 thru 0x508fff (0x7000 bytes) 000431[1] 1748980369.407095 write_OD 0x8ba000 thru 0x8c4fff (0xb000 bytes) 000434[0] 1748980369.421155 read_OD 0x2eb000 thru 0x2f6fff (0xc000 bytes) 000435[0] 1748980369.423627 read_OD 0x393000 thru 0x39cfff (0xa000 bytes) 000441[1] 1748980369.426627 read_OD 0x5c2000 thru 0x5ccfff (0xb000 bytes) 000442[1] 1748980369.428827 read_OD 0x308000 thru 0x315fff (0xe000 bytes) 000443[1] 1748980369.431151 write_OD 0x884000 thru 0x88dfff (0xa000 bytes) 000445[0] 1748980369.444742 mapwrite 0x500000 thru 0x50b1fc (0xb1fd bytes) 000447[1] 1748980369.489429 read_OD 0x505000 thru 0x508fff (0x4000 bytes) 000448[1] 1748980369.501912 mapread 0x6b4000 thru 0x6c3960 (0xf961 bytes) 000449[0] 1748980369.525089 read_OD 0x3f5000 thru 0x401fff (0xd000 bytes) 000450[0] 1748980369.527811 read_OD 0x552000 thru 0x55efff (0xd000 bytes) 000452[1] 1748980369.530553 mapread 0x890000 thru 0x89f50c (0xf50d bytes) 000453[1] 1748980369.530809 read_OD 0x0ab000 thru 0x0b8fff (0xe000 bytes) 000457[1] 1748980369.533861 read_OD 0x0fb000 thru 0x101fff (0x7000 bytes) 000459[1] 1748980369.536532 read_OD 0x171000 thru 0x173fff (0x3000 bytes) 000461[1] 1748980369.538234 write_OD 0x981000 thru 0x98efff (0xe000 bytes) 000462[1] 1748980369.560172 trunc from 0x98f000 to 0x83ac63 (0x15439c bytes) 000463[0] 1748980369.570406 trunc from 0x83ac64 to 0x0945e0 (0x7a6683 bytes) 000464[1] 1748980369.580640 mapread 0x092000 thru 0x093961 (0x1962 bytes) 000466[1] 1748980369.591268 write_OD 0x410000 thru 0x41efff (0xf000 bytes) 000467[0] 1748980369.620714 mapread 0x112000 thru 0x113406 (0x1407 bytes) 000474[1] 1748980369.631146 mapwrite 0x815000 thru 0x81e6a4 (0x96a5 bytes) 000478[0] 1748980369.737865 mapread 0x78d000 thru 0x795406 (0x8407 bytes) 000480[0] 1748980369.783530 mapwrite 0x660000 thru 0x66a036 (0xa037 bytes) 000481[1] 1748980369.862850 mapwrite 0x650000 thru 0x65936a (0x936b bytes) 000482[0] 1748980369.916015 read_OD 0x4e5000 thru 0x4f1fff (0xd000 bytes) 000485[0] 1748980369.928836 trunc from 0x81e6a5 to 0x37012e (0x4ae576 bytes) 000487[1] 1748980369.956942 mapread 0x155000 thru 0x157c49 (0x2c4a bytes) 000488[0] 1748980369.970661 mapread 0x02a000 thru 0x02ad87 (0xd88 bytes) 000490[0] 1748980369.977869 read_OD 0x03c000 thru 0x03efff (0x3000 bytes) 000492[0] 1748980369.980664 mapread 0x1bc000 thru 0x1c06a7 (0x46a8 bytes) 000493[1] 1748980370.012708 read_OD 0x096000 thru 0x09afff (0x5000 bytes) 000496[1] 1748980370.017340 mapwrite 0x7bb000 thru 0x7c2233 (0x7234 bytes) All operations completed A-OK! PASS 16a (12s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 15:52:51 (1748980371) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0700591 s, 29.9 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0563855 s, 37.2 MB/s Chance of close/open is 1 in 50 Seed set to 371 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[1] 1748980371.990182 write 0x9e1857 thru 0x9e2d74 (0x151e bytes) truncating to largest ever: 0x2acf62 000002[1] 1748980371.996112 trunc from 0x9e2d75 to 0x2acf61 (0x735e13 bytes) truncating to largest ever: 0x810944 000004[0] 1748980372.004844 trunc from 0x2acf62 to 0x810943 (0x5639e2 bytes) 000006[0] 1748980372.019518 trunc from 0x810944 to 0x75b037 (0xb590c bytes) 000008[1] 1748980372.030756 write 0x3f6d93 thru 0x3f7362 (0x5d0 bytes) 000011[1] 1748980372.045600 mapwrite 0x2c8d42 thru 0x2c9f1b (0x11da bytes) 000012[0] 1748980372.078501 write 0x07dc6d thru 0x07dda2 (0x136 bytes) 000014[0] 1748980372.089375 read 0x43e1be thru 0x43f815 (0x1658 bytes) 000019[0] 1748980372.108356 trunc from 0x75b038 to 0x502fe2 (0x258055 bytes) 000020[1] 1748980372.117358 read 0x008d5c thru 0x0094bd (0x762 bytes) 000021[0] 1748980372.123436 mapwrite 0x267209 thru 0x2677a0 (0x598 bytes) 000024[1] 1748980372.153222 read 0x069b2e thru 0x06acca (0x119d bytes) 000027[1] 1748980372.170540 write 0x91100f thru 0x911310 (0x302 bytes) 000029[0] 1748980372.180600 read 0x2fa838 thru 0x2fbe6f (0x1638 bytes) 000031[0] 1748980372.186482 write 0x860801 thru 0x861007 (0x807 bytes) 000035[1] 1748980372.203345 trunc from 0x911311 to 0x0632ee (0x8ae022 bytes) 000036[1] 1748980372.211645 write 0x226067 thru 0x227448 (0x13e2 bytes) 000037[1] 1748980372.212308 write 0x06520b thru 0x066aa1 (0x1897 bytes) 000038[1] 1748980372.216946 write 0x2e9073 thru 0x2e95fb (0x589 bytes) 000039[0] 1748980372.223569 read 0x2b7910 thru 0x2b98db (0x1fcc bytes) 000040[1] 1748980372.235116 mapwrite 0x36aa51 thru 0x36b791 (0xd41 bytes) 000041[1] 1748980372.266947 trunc from 0x36b792 to 0x104524 (0x26726d bytes) 000042[0] 1748980372.272928 read 0x0acda2 thru 0x0ae5a8 (0x1807 bytes) 000044[1] 1748980372.284916 write 0x0fbb3f thru 0x0fd8fc (0x1dbe bytes) 000045[1] 1748980372.292426 write 0x8c486b thru 0x8c61f0 (0x1986 bytes) truncating to largest ever: 0x9ea6f6 000046[1] 1748980372.294797 trunc from 0x8c61f1 to 0x9ea6f5 (0x124505 bytes) 000048[0] 1748980372.299934 read 0x450df4 thru 0x45207f (0x128c bytes) 000050[1] 1748980372.313224 trunc from 0x9ea6f6 to 0x48fdc9 (0x55a92c bytes) 000053[0] 1748980372.329937 mapread 0x2cd458 thru 0x2ce3fc (0xfa5 bytes) 000057[0] 1748980372.344886 mapwrite 0x0b9f03 thru 0x0babd6 (0xcd4 bytes) 000058[1] 1748980372.366135 trunc from 0x48fdca to 0x7f91f5 (0x36942c bytes) 000059[1] 1748980372.375905 trunc from 0x7f91f6 to 0x9e1693 (0x1e849e bytes) 000063[1] 1748980372.380783 read 0x8d44e6 thru 0x8d4995 (0x4b0 bytes) 000064[1] 1748980372.385798 write 0x94df30 thru 0x94f8c3 (0x1994 bytes) 000066[1] 1748980372.388720 mapread 0x57126f thru 0x571adf (0x871 bytes) 000067[1] 1748980372.390508 read 0x6e0f21 thru 0x6e14ba (0x59a bytes) 000068[1] 1748980372.405779 mapwrite 0x2d5454 thru 0x2d7200 (0x1dad bytes) 000069[0] 1748980372.422059 mapwrite 0x22772a thru 0x228ad8 (0x13af bytes) 000070[0] 1748980372.444524 mapwrite 0x19ee40 thru 0x1a0790 (0x1951 bytes) 000071[1] 1748980372.486680 mapread 0x224f4d thru 0x22594d (0xa01 bytes) 000073[1] 1748980372.497170 read 0x35bff3 thru 0x35c97d (0x98b bytes) 000074[0] 1748980372.528611 read 0x0417cc thru 0x042bc2 (0x13f7 bytes) 000076[1] 1748980372.563036 mapread 0x28722e thru 0x2891d4 (0x1fa7 bytes) 000077[0] 1748980372.567530 mapread 0x2a2249 thru 0x2a28d3 (0x68b bytes) 000078[0] 1748980372.568050 mapwrite 0x1d6abc thru 0x1d6e4a (0x38f bytes) 000079[1] 1748980372.602255 mapread 0x7a9654 thru 0x7aaa84 (0x1431 bytes) 000080[1] 1748980372.610005 write 0x83dbbc thru 0x83e3e4 (0x829 bytes) 000081[1] 1748980372.619601 write 0x4a355e thru 0x4a5221 (0x1cc4 bytes) 000086[1] 1748980372.631921 read 0x4cff1f thru 0x4d1e3e (0x1f20 bytes) 000087[0] 1748980372.650816 mapwrite 0x1896e3 thru 0x189e80 (0x79e bytes) 000088[0] 1748980372.672357 trunc from 0x9e1694 to 0x77ca7a (0x264c19 bytes) 000089[0] 1748980372.683522 trunc from 0x77ca7b to 0x918bfe (0x19c184 bytes) 000090[1] 1748980372.687086 read 0x550bfb thru 0x550ffb (0x401 bytes) 000091[1] 1748980372.698781 mapread 0x1d3896 thru 0x1d4bb2 (0x131d bytes) 000092[1] 1748980372.703843 mapwrite 0x0bee88 thru 0x0c06e1 (0x185a bytes) 000094[1] 1748980372.741095 trunc from 0x918bff to 0x19620b (0x7829f3 bytes) 000096[0] 1748980372.753059 mapread 0x0f2c6f thru 0x0f41eb (0x157d bytes) 000098[0] 1748980372.767531 write 0x74aafb thru 0x74b391 (0x897 bytes) 000099[1] 1748980372.775096 trunc from 0x74b392 to 0x56ac0c (0x1e0785 bytes) 000100[0] 1748980372.785928 write 0x914058 thru 0x915637 (0x15e0 bytes) 000104[1] 1748980372.804139 read 0x7a4f15 thru 0x7a50f9 (0x1e5 bytes) 000109[0] 1748980372.823892 trunc from 0x915638 to 0x1ad806 (0x767e31 bytes) 000112[1] 1748980372.842946 write 0x14d03c thru 0x14e74f (0x1714 bytes) 000113[1] 1748980372.851114 trunc from 0x1ad807 to 0x89b0b1 (0x6ed8ab bytes) 000114[0] 1748980372.862978 trunc from 0x89b0b2 to 0x3fab6e (0x4a0543 bytes) 000115[0] 1748980372.874873 read 0x1efa03 thru 0x1f1165 (0x1763 bytes) 000116[0] 1748980372.889600 write 0x9f6ee8 thru 0x9f72bd (0x3d6 bytes) 000117[1] 1748980372.892345 trunc from 0x9f72be to 0x731a4c (0x2c5871 bytes) 000119[1] 1748980372.910936 read 0x33085d thru 0x3316b9 (0xe5d bytes) 000124[1] 1748980372.942387 mapwrite 0x37263c thru 0x3730a2 (0xa67 bytes) 000125[0] 1748980372.956873 mapwrite 0x70afb1 thru 0x70aff5 (0x45 bytes) 000126[1] 1748980372.980765 read 0x623858 thru 0x6255ba (0x1d63 bytes) 000130[1] 1748980372.994713 trunc from 0x731a4d to 0x5bf547 (0x172505 bytes) 000136[1] 1748980373.009510 mapwrite 0x9d5a8f thru 0x9d66d5 (0xc47 bytes) 000137[1] 1748980373.034984 mapread 0x23a486 thru 0x23c098 (0x1c13 bytes) 000138[0] 1748980373.039895 mapwrite 0x98a6c5 thru 0x98c523 (0x1e5f bytes) 000140[0] 1748980373.067004 write 0x996c5a thru 0x998949 (0x1cf0 bytes) 000142[1] 1748980373.077506 write 0x88741b thru 0x889309 (0x1eef bytes) 000145[1] 1748980373.093167 trunc from 0x9d66d6 to 0x4a0316 (0x5363bf bytes) 000146[1] 1748980373.102471 mapwrite 0x4023d0 thru 0x402703 (0x334 bytes) 000151[0] 1748980373.122635 read 0x256c07 thru 0x258832 (0x1c2c bytes) 000152[0] 1748980373.133541 read 0x491d1a thru 0x492c4c (0xf33 bytes) 000153[1] 1748980373.136195 read 0x0155cc thru 0x017139 (0x1b6e bytes) 000155[1] 1748980373.158196 mapwrite 0x00288d thru 0x003408 (0xb7c bytes) 000157[0] 1748980373.189981 write 0x540fb9 thru 0x541a37 (0xa7f bytes) 000158[0] 1748980373.198018 mapwrite 0x2794f4 thru 0x27aa1e (0x152b bytes) 000159[0] 1748980373.221467 mapwrite 0x257f11 thru 0x259688 (0x1778 bytes) 000160[0] 1748980373.238896 read 0x2b0453 thru 0x2b1ad9 (0x1687 bytes) 000161[1] 1748980373.251174 trunc from 0x541a38 to 0x2c01f5 (0x281842 bytes) 000162[1] 1748980373.264755 read 0x1374ab thru 0x13937c (0x1ed2 bytes) 000163[1] 1748980373.281079 mapwrite 0x62cd63 thru 0x62e6b3 (0x1951 bytes) 000166[1] 1748980373.315392 trunc from 0x62e6b4 to 0x39c815 (0x291e9e bytes) 000167[1] 1748980373.321099 write 0x9e493e thru 0x9e58e7 (0xfaa bytes) 000168[1] 1748980373.323134 mapread 0x72460a thru 0x724654 (0x4b bytes) 000169[1] 1748980373.339458 write 0x7e14fd thru 0x7e253c (0x1040 bytes) 000170[1] 1748980373.344524 trunc from 0x9e58e8 to 0x294cf0 (0x750bf7 bytes) 000172[1] 1748980373.360663 mapwrite 0x47a66f thru 0x47b5ef (0xf81 bytes) 000175[1] 1748980373.384892 write 0x61a3ac thru 0x61b727 (0x137c bytes) 000176[0] 1748980373.389036 write 0x64500c thru 0x64695c (0x1951 bytes) 000178[0] 1748980373.401186 trunc from 0x64695d to 0x8b3ce6 (0x26d38a bytes) 000181[1] 1748980373.417701 read 0x000c3b thru 0x001349 (0x70f bytes) 000183[0] 1748980373.431183 trunc from 0x8b3ce7 to 0x5a8624 (0x30b6c2 bytes) 000186[1] 1748980373.446805 read 0x4c77cf thru 0x4c835e (0xb90 bytes) 000189[1] 1748980373.461960 mapwrite 0x9f2b76 thru 0x9f3519 (0x9a4 bytes) 000190[1] 1748980373.506893 write 0x1db141 thru 0x1dbf07 (0xdc7 bytes) 000192[1] 1748980373.515060 trunc from 0x9f351a to 0x7456f9 (0x2ade20 bytes) 000194[1] 1748980373.521751 read 0x480ffd thru 0x481531 (0x535 bytes) 000196[0] 1748980373.533125 read 0x453c6b thru 0x455bd2 (0x1f68 bytes) 000197[0] 1748980373.548349 trunc from 0x7456fa to 0x4ef2f4 (0x256405 bytes) 000198[1] 1748980373.565662 read 0x00f7b9 thru 0x0113d5 (0x1c1d bytes) 000200[0] 1748980373.578996 trunc from 0x4ef2f5 to 0x5dcfc6 (0xedcd2 bytes) 000201[0] 1748980373.590993 mapwrite 0x597367 thru 0x598280 (0xf1a bytes) 000202[0] 1748980373.615201 read 0x2a35e9 thru 0x2a46f4 (0x110c bytes) 000203[0] 1748980373.628738 mapwrite 0x8accc5 thru 0x8adef7 (0x1233 bytes) 000204[0] 1748980373.663499 write 0x1ce23f thru 0x1ce282 (0x44 bytes) 000205[1] 1748980373.668311 write 0x7407f6 thru 0x74276f (0x1f7a bytes) 000207[1] 1748980373.691130 mapread 0x106b38 thru 0x106db1 (0x27a bytes) 000210[1] 1748980373.703795 read 0x0c4a13 thru 0x0c5587 (0xb75 bytes) 000211[0] 1748980373.735789 mapwrite 0x18a7a7 thru 0x18b10c (0x966 bytes) 000212[1] 1748980373.761369 read 0x11b36b thru 0x11bf9f (0xc35 bytes) 000213[0] 1748980373.781391 mapread 0x2a8524 thru 0x2a8921 (0x3fe bytes) 000217[0] 1748980373.784542 write 0x3dc550 thru 0x3de2f7 (0x1da8 bytes) 000218[0] 1748980373.791555 read 0x784e92 thru 0x7858ee (0xa5d bytes) 000221[1] 1748980373.800749 read 0x340df6 thru 0x342179 (0x1384 bytes) 000223[1] 1748980373.821499 read 0x85a4a1 thru 0x85b4e1 (0x1041 bytes) 000225[0] 1748980373.821845 trunc from 0x8adef8 to 0x8337a0 (0x7a757 bytes) 000226[1] 1748980373.835956 mapread 0x647c31 thru 0x648dec (0x11bc bytes) 000227[0] 1748980373.846126 trunc from 0x8337a1 to 0x2cdedd (0x5658c3 bytes) 000228[0] 1748980373.852040 read 0x0c18ca thru 0x0c3415 (0x1b4c bytes) 000229[1] 1748980373.860466 trunc from 0x2cdede to 0x75b29a (0x48d3bd bytes) 000231[0] 1748980373.870857 trunc from 0x75b29b to 0x274eff (0x4e639b bytes) 000232[0] 1748980373.877887 write 0x2b4ade thru 0x2b5397 (0x8ba bytes) 000237[1] 1748980373.894901 trunc from 0x2b5398 to 0x28c4f0 (0x28ea7 bytes) 000238[0] 1748980373.904229 mapwrite 0x71fd89 thru 0x721d03 (0x1f7b bytes) 000240[1] 1748980373.951679 trunc from 0x721d04 to 0x33449e (0x3ed865 bytes) 000242[0] 1748980373.972499 mapread 0x12fd86 thru 0x130716 (0x991 bytes) 000243[1] 1748980373.980766 mapwrite 0x093c3e thru 0x093faa (0x36d bytes) 000244[0] 1748980374.000594 write 0x114b37 thru 0x11638b (0x1855 bytes) 000245[0] 1748980374.007718 write 0x4518ae thru 0x4519ef (0x142 bytes) 000246[1] 1748980374.010430 read 0x0c0ef4 thru 0x0c144a (0x557 bytes) 000249[1] 1748980374.019253 mapwrite 0x5829c3 thru 0x5830f0 (0x72e bytes) 000250[1] 1748980374.066708 mapread 0x0bf238 thru 0x0bfb9e (0x967 bytes) 000251[0] 1748980374.072851 write 0x97981a thru 0x979f7d (0x764 bytes) 000252[1] 1748980374.085197 mapwrite 0x92ec72 thru 0x92fe6d (0x11fc bytes) 000254[1] 1748980374.116479 mapread 0x7dd778 thru 0x7de7e8 (0x1071 bytes) 000255[1] 1748980374.121190 mapread 0x0ae573 thru 0x0afee6 (0x1974 bytes) 000256[1] 1748980374.123725 read 0x23fab3 thru 0x23fc00 (0x14e bytes) 000257[1] 1748980374.129592 mapwrite 0x23ba2d thru 0x23d5f4 (0x1bc8 bytes) 000264[0] 1748980374.160134 mapwrite 0x52b6d1 thru 0x52d174 (0x1aa4 bytes) 000265[1] 1748980374.186649 read 0x022d6f thru 0x023b51 (0xde3 bytes) 000267[0] 1748980374.201538 mapread 0x52a991 thru 0x52aae1 (0x151 bytes) 000269[1] 1748980374.211749 read 0x336aca thru 0x33714e (0x685 bytes) 000270[0] 1748980374.239981 write 0x1d3b32 thru 0x1d59d6 (0x1ea5 bytes) 000271[0] 1748980374.251558 trunc from 0x979f7e to 0x6b931c (0x2c0c61 bytes) 000272[0] 1748980374.268214 mapwrite 0x5895a6 thru 0x58a979 (0x13d4 bytes) 000274[1] 1748980374.288625 trunc from 0x6b931d to 0x8c5311 (0x20bff5 bytes) 000276[1] 1748980374.297704 mapwrite 0x55eff2 thru 0x560cfd (0x1d0c bytes) 000280[0] 1748980374.318772 write 0x915efa thru 0x91752d (0x1634 bytes) 000282[0] 1748980374.328455 write 0x6e897d thru 0x6ea92c (0x1fb0 bytes) 000283[1] 1748980374.335704 write 0x900d99 thru 0x9011bc (0x424 bytes) 000284[0] 1748980374.344369 trunc from 0x91752e to 0x1ab45d (0x76c0d0 bytes) 000285[1] 1748980374.355633 mapread 0x0147dd thru 0x01592e (0x1152 bytes) 000287[1] 1748980374.367137 mapwrite 0x4e1f9a thru 0x4e2c23 (0xc8a bytes) 000288[1] 1748980374.396078 read 0x21a0ac thru 0x21ada1 (0xcf6 bytes) 000289[1] 1748980374.416592 write 0x4fb67e thru 0x4fbf40 (0x8c3 bytes) 000290[0] 1748980374.417070 write 0x6216f2 thru 0x62253e (0xe4d bytes) 000298[1] 1748980374.450513 read 0x54c557 thru 0x54daf6 (0x15a0 bytes) 000300[1] 1748980374.462585 mapwrite 0x36148d thru 0x361f2a (0xa9e bytes) 000302[0] 1748980374.483350 trunc from 0x62253f to 0x5326df (0xefe5f bytes) 000305[0] 1748980374.499500 trunc from 0x5326e0 to 0x499e22 (0x988bd bytes) 000306[0] 1748980374.504161 trunc from 0x499e23 to 0x34f868 (0x14a5ba bytes) 000307[0] 1748980374.507748 trunc from 0x34f869 to 0x8493de (0x4f9b76 bytes) 000308[0] 1748980374.512113 read 0x355ff0 thru 0x356956 (0x967 bytes) 000310[1] 1748980374.533733 mapread 0x2cbd7c thru 0x2cc6ea (0x96f bytes) 000315[1] 1748980374.557209 mapwrite 0x380ea5 thru 0x382b91 (0x1ced bytes) 000316[1] 1748980374.577737 write 0x449b74 thru 0x449f24 (0x3b1 bytes) 000320[0] 1748980374.588542 write 0x6a055a thru 0x6a2442 (0x1ee9 bytes) 000321[1] 1748980374.597319 mapwrite 0x0594f1 thru 0x05a1c4 (0xcd4 bytes) 000324[0] 1748980374.617488 trunc from 0x8493df to 0x21f08c (0x62a352 bytes) 000325[0] 1748980374.628094 trunc from 0x21f08d to 0x6f6278 (0x4d71ec bytes) 000327[0] 1748980374.640327 trunc from 0x6f6279 to 0x0a9f71 (0x64c307 bytes) 000332[1] 1748980374.649802 write 0x373475 thru 0x3737b1 (0x33d bytes) 000335[1] 1748980374.663313 trunc from 0x3737b2 to 0x2e15c0 (0x921f1 bytes) 000338[0] 1748980374.677978 mapread 0x11adb5 thru 0x11bba2 (0xdee bytes) 000341[0] 1748980374.688442 mapread 0x219e86 thru 0x21ae4a (0xfc5 bytes) 000344[0] 1748980374.695002 read 0x175edc thru 0x176240 (0x365 bytes) 000345[0] 1748980374.701389 trunc from 0x2e15c1 to 0x47249d (0x190edd bytes) 000346[0] 1748980374.710739 mapwrite 0x53d825 thru 0x53dc54 (0x430 bytes) 000347[0] 1748980374.738627 read 0x4a5a80 thru 0x4a60c5 (0x646 bytes) 000349[1] 1748980374.743718 mapwrite 0x1b660c thru 0x1b78d1 (0x12c6 bytes) 000351[1] 1748980374.768968 mapread 0x08d002 thru 0x08d7b8 (0x7b7 bytes) 000352[0] 1748980374.778994 write 0x118c90 thru 0x11903a (0x3ab bytes) 000353[0] 1748980374.783466 write 0x4e94ce thru 0x4e987c (0x3af bytes) 000354[1] 1748980374.786691 mapread 0x3fd59b thru 0x3ff4f1 (0x1f57 bytes) 000355[1] 1748980374.797303 mapread 0x0a1b7f thru 0x0a2e82 (0x1304 bytes) 000358[0] 1748980374.805573 trunc from 0x53dc55 to 0x7b96e3 (0x27ba8f bytes) 000359[1] 1748980374.814364 mapread 0x0b7d25 thru 0x0b8756 (0xa32 bytes) 000361[0] 1748980374.828420 mapwrite 0x7d50b9 thru 0x7d57d4 (0x71c bytes) 000364[0] 1748980374.851383 trunc from 0x7d57d5 to 0x01e32f (0x7b74a5 bytes) 000365[0] 1748980374.855802 read 0x01d2a2 thru 0x01d698 (0x3f7 bytes) 000367[0] 1748980374.862450 read 0x018649 thru 0x018e57 (0x80f bytes) 000369[0] 1748980374.864692 write 0x2c9a64 thru 0x2caec5 (0x1462 bytes) 000374[1] 1748980374.875905 write 0x190f81 thru 0x192f1e (0x1f9e bytes) 000376[0] 1748980374.884670 mapread 0x14aa1b thru 0x14aa31 (0x17 bytes) 000377[1] 1748980374.904084 trunc from 0x2caec6 to 0x185179 (0x145d4c bytes) 000378[1] 1748980374.919511 mapread 0x0b8a7d thru 0x0ba5c1 (0x1b45 bytes) 000379[0] 1748980374.924616 write 0x4282f3 thru 0x429254 (0xf62 bytes) 000380[0] 1748980374.930856 trunc from 0x429255 to 0x20d8da (0x21b97a bytes) 000384[1] 1748980374.945218 trunc from 0x20d8db to 0x2c488b (0xb6fb1 bytes) 000385[0] 1748980374.952675 mapwrite 0x023999 thru 0x024bd1 (0x1239 bytes) 000386[0] 1748980374.974882 write 0x43a08e thru 0x43aab5 (0xa28 bytes) 000387[0] 1748980374.979293 mapwrite 0x6cb05a thru 0x6cb2d2 (0x279 bytes) 000391[0] 1748980375.027561 mapwrite 0x0bcd2a thru 0x0beb4a (0x1e21 bytes) 000393[1] 1748980375.052285 trunc from 0x6cb2d3 to 0x36ac34 (0x36069e bytes) 000394[1] 1748980375.064848 trunc from 0x36ac35 to 0x6200d7 (0x2b54a3 bytes) 000395[0] 1748980375.072071 write 0x1ecc8d thru 0x1ee149 (0x14bd bytes) 000396[0] 1748980375.080974 read 0x1867ef thru 0x1879cb (0x11dd bytes) 000397[1] 1748980375.089675 mapread 0x5120ea thru 0x513831 (0x1748 bytes) 000398[0] 1748980375.100681 trunc from 0x6200d8 to 0x4bea5c (0x16167b bytes) 000399[1] 1748980375.110989 mapread 0x443ff1 thru 0x445b2f (0x1b3f bytes) 000401[1] 1748980375.123737 read 0x2cd605 thru 0x2cf512 (0x1f0e bytes) 000402[0] 1748980375.127343 read 0x0b8c70 thru 0x0b91bd (0x54e bytes) 000404[1] 1748980375.149380 mapread 0x364432 thru 0x3650e3 (0xcb2 bytes) 000407[1] 1748980375.167479 write 0x0a06fe thru 0x0a09e8 (0x2eb bytes) 000408[0] 1748980375.175963 mapread 0x153dce thru 0x15421c (0x44f bytes) 000409[0] 1748980375.183361 mapwrite 0x7e72c0 thru 0x7e7e2c (0xb6d bytes) 000413[0] 1748980375.229720 write 0x60c4b5 thru 0x60d4a3 (0xfef bytes) 000414[0] 1748980375.233625 read 0x198512 thru 0x19985a (0x1349 bytes) 000415[1] 1748980375.248454 mapread 0x53a3db thru 0x53ab05 (0x72b bytes) 000418[0] 1748980375.263120 mapwrite 0x08d4c6 thru 0x08e3f8 (0xf33 bytes) 000419[1] 1748980375.280678 mapread 0x77219d thru 0x772a54 (0x8b8 bytes) 000423[0] 1748980375.292720 trunc from 0x7e7e2d to 0x40648f (0x3e199d bytes) 000424[1] 1748980375.305224 read 0x3b9594 thru 0x3baf9c (0x1a09 bytes) 000426[1] 1748980375.318823 mapread 0x3a553a thru 0x3a6d59 (0x1820 bytes) 000427[0] 1748980375.325860 mapread 0x24bc35 thru 0x24bd2a (0xf6 bytes) 000428[1] 1748980375.326213 mapwrite 0x33eff3 thru 0x340cad (0x1cbb bytes) 000429[0] 1748980375.349024 read 0x07b51f thru 0x07c8a4 (0x1386 bytes) 000432[1] 1748980375.367129 mapwrite 0x9db042 thru 0x9db0f3 (0xb2 bytes) 000433[1] 1748980375.410956 write 0x3e84b0 thru 0x3ea438 (0x1f89 bytes) 000435[0] 1748980375.418687 read 0x354ad9 thru 0x3564d8 (0x1a00 bytes) 000437[0] 1748980375.438326 mapread 0x1d197c thru 0x1d2a06 (0x108b bytes) 000438[0] 1748980375.446666 mapread 0x3dd384 thru 0x3de42c (0x10a9 bytes) 000443[0] 1748980375.463698 write 0x14f11e thru 0x1507ef (0x16d2 bytes) 000445[1] 1748980375.477054 mapread 0x176dbe thru 0x1779e3 (0xc26 bytes) 000447[0] 1748980375.487975 mapread 0x6852da thru 0x686a5b (0x1782 bytes) 000449[0] 1748980375.497562 mapwrite 0x8e91fd thru 0x8ea1a4 (0xfa8 bytes) 000453[1] 1748980375.514861 write 0x583355 thru 0x584ab6 (0x1762 bytes) 000455[0] 1748980375.526233 read 0x5fc46c thru 0x5fc4cc (0x61 bytes) 000456[0] 1748980375.550099 mapwrite 0x8d6bfe thru 0x8d76b5 (0xab8 bytes) 000457[0] 1748980375.566707 write 0x3c6009 thru 0x3c64f1 (0x4e9 bytes) 000458[0] 1748980375.572391 trunc from 0x9db0f4 to 0x134270 (0x8a6e83 bytes) 000459[1] 1748980375.583135 read 0x0d3cf0 thru 0x0d418c (0x49d bytes) 000460[1] 1748980375.590285 read 0x05ef14 thru 0x05f1b8 (0x2a5 bytes) 000465[1] 1748980375.599557 trunc from 0x134271 to 0x6d3c81 (0x59fa11 bytes) 000466[1] 1748980375.609473 mapread 0x5d0f2c thru 0x5d1eb6 (0xf8b bytes) 000468[0] 1748980375.643332 mapwrite 0x2588ed thru 0x2594f8 (0xc0c bytes) 000469[1] 1748980375.668802 read 0x47a035 thru 0x47beca (0x1e96 bytes) 000471[0] 1748980375.683365 mapwrite 0x18159c thru 0x18306d (0x1ad2 bytes) 000476[0] 1748980375.715551 trunc from 0x6d3c82 to 0x65d710 (0x76571 bytes) 000479[1] 1748980375.728530 mapwrite 0x882bd1 thru 0x884b8f (0x1fbf bytes) 000483[0] 1748980375.767832 write 0x186cdb thru 0x1878fb (0xc21 bytes) 000484[1] 1748980375.776363 write 0x4a1b30 thru 0x4a298b (0xe5c bytes) 000485[1] 1748980375.782503 read 0x7b7767 thru 0x7b9523 (0x1dbd bytes) 000486[0] 1748980375.790653 write 0x5847f7 thru 0x584f23 (0x72d bytes) 000487[0] 1748980375.797642 trunc from 0x884b90 to 0x84bbc6 (0x38fc9 bytes) 000488[1] 1748980375.809385 trunc from 0x84bbc7 to 0x619313 (0x2328b3 bytes) 000489[0] 1748980375.817596 mapwrite 0x6a871d thru 0x6a9006 (0x8ea bytes) 000491[0] 1748980375.848208 mapread 0x455140 thru 0x456ad3 (0x1994 bytes) 000494[1] 1748980375.851186 mapwrite 0x810d19 thru 0x811049 (0x331 bytes) 000495[1] 1748980375.878542 write 0x3f245b thru 0x3f3d43 (0x18e9 bytes) 000496[0] 1748980375.884386 mapwrite 0x9a25d8 thru 0x9a2ffa (0xa23 bytes) 000497[1] 1748980375.913168 mapread 0x8448d5 thru 0x84687e (0x1faa bytes) 000499[1] 1748980375.929257 write 0x5ed23e thru 0x5eddc3 (0xb86 bytes) All operations completed A-OK! PASS 16b (5s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 15:52:57 (1748980377) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 15:52:57 (1748980377) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0609391 s, 34.4 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0532019 s, 39.4 MB/s ldlm.namespaces.MGC192.168.203.154@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff88012bf96000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff88012dbf1800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff88012bf96000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff88012dbf1800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff88012bf96000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.6559 s, 39.5 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.621015 s, 169 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.516085 s, 203 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.20727 s, 86.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.47462 s, 71.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.9217 s, 114 MB/s PASS 16d (11s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 15:53:08 (1748980388) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0552976 s, 190 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276919 s, 37.9 MB/s PASS 16e (1s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 15:53:09 (1748980389) TIMEOUT PASS 16f (22s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 15:53:31 (1748980411) TIMEOUT PASS 16g (22s) == sanityn test 16i: read after truncate file ============ 15:53:53 (1748980433) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (1s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 15:53:54 (1748980434) fail_loc=0x8000030a PASS 17 (4s) == sanityn test 18: mmap sanity check =========================================================================================== 15:53:58 (1748980438) mmap test1: basic mmap operation (PASS, 0.013846s) mmap test2: MAP_PRIVATE not write back (PASS, 0.011731s) mmap test3: concurrent mmap ops on two nodes (PASS, 2.4067s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.0645s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.021908s) mmap test6: check mmap write/read content on two nodes (PASS, 0.038738s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0124s) PASS 18 (8s) == sanityn test 19: test concurrent uncached read races ========================================================================= 15:54:06 (1748980446) oleg354-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg354-client: oleg354-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 15:54:07 (1748980447) PASS 20 (2s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 15:54:09 (1748980449) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (1s) == sanityn test 23: others should see updated atime while another read============================================================== 15:54:10 (1748980450) now is 1748980451 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6866 new atime is 1748980512 PASS 23 (63s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 15:55:13 (1748980513) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 10240 7528448 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 10240 7528448 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.3K 374 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 375 115.0K 1% /mnt/lustre[OST:1] filesystem_summary: 230.2K 368 229.9K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 374 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 375 115.0K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 368 229.9K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 4.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 10.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118038 374 117664 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118103 375 117728 1% /mnt/lustre2[OST:1] filesystem_summary: 235760 368 235392 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 10240 7528448 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.5K 368 488.1K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 374 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 375 115.0K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 368 229.9K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118038 374 117664 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118103 375 117728 1% /mnt/lustre[OST:1] filesystem_summary: 235760 368 235392 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500203 368 499835 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118038 374 117664 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118103 375 117728 1% /mnt/lustre2[OST:1] filesystem_summary: 235760 368 235392 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 10240 7528448 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 4096 3765248 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 10240 7528448 1% /mnt/lustre2 PASS 24a (1s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 15:55:14 (1748980514) PASS 24b (2s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 15:55:16 (1748980516) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (1s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 15:55:17 (1748980517) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 15:55:18 (1748980518) utime: good mknod times 1748980517 <= 1748980518 <= 1748980518 for /mnt/lustre/f26a utime: good mknod times 1748980517 <= 1748980518 <= 1748980518 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (2s) == sanityn test 26b: sync mtime between ost and mds ====== 15:55:20 (1748980520) PASS 26b (4s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 15:55:24 (1748980524) 0 PASS 30 (4s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 15:55:28 (1748980528) fail_loc=0x314 PASS 31a (2s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 15:55:30 (1748980530) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg354-server mds-ost sync done. sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.203.154@tcp:/lustre 7542784 10240 7528448 1% /mnt/lustre2 PASS 31b (18s) == sanityn test 31r: open-rename(replace) race =========== 15:55:48 (1748980548) -rw-r--r-- 1 root root 0 Jun 3 15:55 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (4s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 15:55:53 (1748980553) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (0s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 15:55:53 (1748980553) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 15:55:54 (1748980554) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 15:55:55 (1748980555) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (0s) == sanityn test 34: no lock timeout under IO ============= 15:55:56 (1748980556) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg354-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88012bf96000.ost_server_uuid,osc.lustre-OST0000-osc-ffff88012dbf1800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff88012bf96000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff88012dbf1800.ost_server_uuid in FULL state after 0 sec oleg354-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff88012bf96000.ost_server_uuid,osc.lustre-OST0001-osc-ffff88012dbf1800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff88012bf96000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff88012dbf1800.ost_server_uuid in IDLE state after 0 sec oleg354-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88012bf96000.ost_server_uuid,osc.lustre-OST0000-osc-ffff88012dbf1800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff88012bf96000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff88012dbf1800.ost_server_uuid in FULL state after 0 sec oleg354-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff88012bf96000.ost_server_uuid,osc.lustre-OST0001-osc-ffff88012dbf1800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff88012bf96000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff88012dbf1800.ost_server_uuid in IDLE state after 0 sec oleg354-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88012bf96000.ost_server_uuid,osc.lustre-OST0000-osc-ffff88012dbf1800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff88012bf96000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff88012dbf1800.ost_server_uuid in IDLE state after 0 sec oleg354-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff88012bf96000.ost_server_uuid,osc.lustre-OST0001-osc-ffff88012dbf1800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff88012bf96000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff88012dbf1800.ost_server_uuid in IDLE state after 0 sec PASS 34 (70s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 15:57:06 (1748980626) fail_loc=0x80000317 Race attempt 0 Wait for 15683 15822 for 60 sec... total: 4000 open/close in 7.44 seconds: 537.29 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.203.154@tcp:/lustre 7.2G 12M 7.2G 1% /mnt/lustre 192.168.203.154@tcp:/lustre 7.2G 12M 7.2G 1% /mnt/lustre2 PASS 35 (63s) == sanityn test 36: handle ESTALE/open-unlink correctly == 15:58:09 (1748980689) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg354-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.277746 s, 189 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6866 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.241473 s, 217 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6866 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.292107 s, 179 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6866 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.256231 s, 205 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6866 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.269416 s, 195 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6866 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.283487 s, 185 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6866 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.262941 s, 199 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6866 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.254401 s, 206 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6866 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(7) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.277792 s, 189 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6866 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.26058 s, 201 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6866 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7526400) after_dd(7475200) after(7526400) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.31692 s, 165 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6866 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(10) *** before(7526400) after_dd(7475200) after(7526400) PASS 36 (227s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 16:01:56 (1748980916) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6866 - create 9713 (time 1748980927.84 total 10.00 last 971.27) total: 10000 create in 10.28 seconds: 972.73 ops/second PASS 37 (13s) == sanityn test 39a: test from 11063 ============================================================================================ 16:02:09 (1748980929) repeat after cancel_lru_locks PASS 39a (2s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 16:02:11 (1748980931) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 16:02:14 (1748980934) repeat after cancel_lru_locks PASS 39c (2s) == sanityn test 39d: sync write should update mtime ====== 16:02:16 (1748980936) fail_loc=0x411 fail_loc=0 PASS 39d (2s) == sanityn test 40a: pdirops: create vs others ======================================================================== 16:02:18 (1748980938) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (8s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 16:02:26 (1748980946) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (7s) == sanityn test 40c: pdirops: link and others ======================================================================== 16:02:33 (1748980953) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (7s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 16:02:41 (1748980961) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (8s) == sanityn test 40e: pdirops: rename and others ======================================================================== 16:02:48 (1748980968) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (6s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 16:02:54 (1748980974) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (4s) == sanityn test 41b: pdirops: create vs create ======================================================================== 16:02:58 (1748980978) open: File exists Conflict PASS 41b (5s) == sanityn test 41c: pdirops: create vs link ======================================================================== 16:03:03 (1748980983) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (4s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 16:03:07 (1748980987) Conflict PASS 41d (4s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 16:03:11 (1748980991) Conflict PASS 41e (5s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 16:03:16 (1748980996) Conflict PASS 41f (4s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 16:03:20 (1748981000) Conflict PASS 41g (4s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 16:03:24 (1748981004) Conflict PASS 41h (4s) == sanityn test 41i: reint_open: create vs create ======== 16:03:28 (1748981008) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (676s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 16:14:44 (1748981684) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (4s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 16:14:49 (1748981689) open: File exists Conflict PASS 42b (5s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 16:14:53 (1748981693) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (4s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 16:14:57 (1748981697) Conflict PASS 42d (4s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 16:15:01 (1748981701) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (4s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 16:15:05 (1748981705) Conflict PASS 42f (4s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 16:15:09 (1748981709) Conflict PASS 42g (5s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 16:15:14 (1748981714) Conflict PASS 42h (4s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 16:15:18 (1748981718) PASS 43a (11s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 16:15:29 (1748981729) Conflict PASS 43b (4s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 16:15:33 (1748981733) Conflict PASS 43c (5s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 16:15:38 (1748981738) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (4s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 16:15:42 (1748981742) Conflict PASS 43e (4s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 16:15:46 (1748981746) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (4s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 16:15:50 (1748981750) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (4s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 16:15:54 (1748981754) Conflict PASS 43h (4s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 16:15:58 (1748981758) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 16:15:59 (1748981759) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (31s) == sanityn test 43k: unlink vs create ==================== 16:16:30 (1748981790) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (357s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 16:22:27 (1748982147) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Tue Jun 3 16:22:30 EDT 2025 PASS 44a (4s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 16:22:31 (1748982151) open: File exists Conflict PASS 44b (4s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 16:22:35 (1748982155) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (5s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 16:22:40 (1748982160) Conflict PASS 44d (4s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 16:22:44 (1748982164) Conflict PASS 44e (4s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 16:22:48 (1748982168) Conflict PASS 44f (4s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 16:22:52 (1748982172) Conflict PASS 44g (5s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 16:22:57 (1748982177) Conflict PASS 44h (4s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 16:23:01 (1748982181) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 16:23:02 (1748982182) PASS 45a (22s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 16:23:24 (1748982204) Conflict PASS 45b (4s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 16:23:28 (1748982208) Conflict PASS 45c (4s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 16:23:32 (1748982212) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (4s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 16:23:36 (1748982216) Conflict PASS 45e (4s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 16:23:40 (1748982220) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (4s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 16:23:44 (1748982224) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (4s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 16:23:48 (1748982228) Conflict PASS 45h (4s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 16:23:52 (1748982232) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (0s) == sanityn test 45j: read vs rename ====================== 16:23:52 (1748982232) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (364s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 16:29:56 (1748982596) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (4s) == sanityn test 46b: pdirops: link vs create ======================================================================== 16:30:00 (1748982600) open: File exists Conflict PASS 46b (5s) == sanityn test 46c: pdirops: link vs link ======================================================================== 16:30:05 (1748982605) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (4s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 16:30:09 (1748982609) Conflict PASS 46d (4s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 16:30:13 (1748982613) Conflict PASS 46e (5s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 16:30:18 (1748982618) Conflict PASS 46f (4s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 16:30:22 (1748982622) Conflict PASS 46g (4s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 16:30:26 (1748982626) Conflict PASS 46h (4s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 16:30:30 (1748982630) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 16:30:31 (1748982631) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 16:30:32 (1748982632) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 16:30:33 (1748982633) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 16:30:34 (1748982634) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 16:30:35 (1748982635) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (0s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 16:30:36 (1748982636) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (0s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 16:30:36 (1748982636) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 16:30:37 (1748982637) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.00418977 s, 2.4 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 16:30:44 (1748982644) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00353548 s, 45.0 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (24635) - No such process PASS 51a (3s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 16:30:47 (1748982647) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00249623 s, 410 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152208 s, 68.9 MB/s 3145728 PASS 51b (14s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 16:31:01 (1748982661) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00745651 s, 137 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 16:31:07 (1748982667) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106621 s, 98.3 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (3s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 16:31:10 (1748982670) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2433 0x981 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2433 0x981 0 read lease(1) has applied. PASS 51e (5s) == sanityn test 54: rename locking ======================= 16:31:15 (1748982675) llite.lustre-ffff88012bf96000.xattr_cache=1 llite.lustre-ffff88012dbf1800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504963 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-06-03 16:31:15.000000000 -0400 Modify: 2025-06-03 16:31:15.000000000 -0400 Change: 2025-06-03 16:31:15.000000000 -0400 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504966 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-06-03 16:31:21.000000000 -0400 Modify: 2025-06-03 16:31:21.000000000 -0400 Change: 2025-06-03 16:31:21.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504969 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-06-03 16:31:26.000000000 -0400 Modify: 2025-06-03 16:31:26.000000000 -0400 Change: 2025-06-03 16:31:26.000000000 -0400 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-06-03 16:31:26.000000000 -0400 Modify: 2025-06-03 16:31:26.000000000 -0400 Change: 2025-06-03 16:31:26.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-06-03 16:31:32.000000000 -0400 Modify: 2025-06-03 16:31:32.000000000 -0400 Change: 2025-06-03 16:31:32.000000000 -0400 Birth: - File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504972 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-06-03 16:31:32.000000000 -0400 Modify: 2025-06-03 16:31:32.000000000 -0400 Change: 2025-06-03 16:31:32.000000000 -0400 Birth: - PASS 54 (24s) == sanityn test 55a: rename vs unlink target dir ========= 16:31:39 (1748982699) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 16:31:46 (1748982706) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 16:31:53 (1748982713) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 16:32:05 (1748982725) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) == sanityn test 60: Verify data_version behaviour ======== 16:32:14 (1748982734) PASS 60 (2s) == sanityn test 70a: cd directory && rm directory ======== 16:32:16 (1748982736) PASS 70a (2s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 16:32:18 (1748982738) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 16:32:20 (1748982740) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00264232 s, 24.8 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (0s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 16:32:21 (1748982741) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00306105 s, 21.4 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (0s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 16:32:21 (1748982741) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 16:32:22 (1748982742) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00339545 s, 19.3 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 16:32:23 (1748982743) llite.lustre-ffff88012bf96000.xattr_cache=1 llite.lustre-ffff88012dbf1800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (1s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 16:32:24 (1748982744) llite.lustre-ffff88012bf96000.xattr_cache=1 llite.lustre-ffff88012dbf1800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (2s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 16:32:26 (1748982746) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00284596 s, 360 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00524813 s, 195 kB/s 3287: get lock2 3288: get lock1 3287: done 3287: get lock1 3288: done 3288: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 3287: cannot get lock1: Resource deadlock avoided fcntl cmd 7 failed: Resource deadlock avoided 3288: cannot get lock2: Resource deadlock avoided 3288: exit rc=0 3287: done 3287: exit rc=0 PASS 74 (5s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 16:32:31 (1748982751) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0131231 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00629799 s, 166 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 3923 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 16:32:38 (1748982758) opening files: ........ closing files: ........ PASS 76 (13s) == sanityn test 77a: check FIFO NRS policy =============== 16:32:51 (1748982771) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0829575 s, 202 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00702487 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00773146 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118201 s, 88.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209729 s, 50.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206957 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00943205 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00841549 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132706 s, 79.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212428 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297484 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277747 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.028655 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392435 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288697 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021301 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364583 s, 28.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012dbf1800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00709384 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00791361 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00389266 s, 269 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103625 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00751825 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00927836 s, 113 MB/s 1048576 bytes (1.0 MB) copied, 0.0121221 s, 86.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00742399 s, 141 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00758224 s, 138 MB/s , 0.00756372 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0074938 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00943203 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107299 s, 97.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105393 s, 99.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104019 s, 101 MB/s , 0.0114102 s, 91.9 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=1 removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg354-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (3s) == sanityn test 77b: check CRR-N NRS policy ============== 16:32:54 (1748982774) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.106295 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00817373 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014503 s, 72.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0204801 s, 51.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226378 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0252714 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222892 s, 47.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233703 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261173 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236647 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202917 s, 51.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328027 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254567 s, 41.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0377229 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182729 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242155 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198937 s, 52.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012dbf1800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00748823 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0091842 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119468 s, 87.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00657461 s, 159 MB/s , 0.0105758 s, 99.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00801564 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147081 s, 71.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00915162 s, 115 MB/s 1048576 bytes (1.0 MB) copied, 0.00186382 s, 563 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114708 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00163817 s, 640 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00189893 s, 552 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=2 1048576 bytes (1.0 MB) copied, 0.00308694 s, 340 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127757 s, 82.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0015995 s, 656 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127044 s, 82.5 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg354-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0976842 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00750062 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156208 s, 67.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117476 s, 89.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0215316 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128959 s, 81.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206556 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266027 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246931 s, 42.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00825479 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214333 s, 48.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249345 s, 42.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140071 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235884 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0097999 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0262859 s, 39.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012dbf1800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025976 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00174881 s, 600 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00778235 s, 135 MB/s , 0.00161179 s, 651 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00496862 s, 211 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101886 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00651893 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00680536 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00732844 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0072443 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00899141 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00905962 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00945944 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010782 s, 97.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011768 s, 89.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119208 s, 88.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120938 s, 86.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=1 removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg354-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) == sanityn test 77c: check ORR NRS policy ================ 16:32:57 (1748982777) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.143642 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118539 s, 88.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218995 s, 47.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241752 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167822 s, 62.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243645 s, 43.0 MB/s , 0.0115151 s, 91.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215303 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259851 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0299148 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255263 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281095 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290269 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0414532 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249618 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249685 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0376522 s, 27.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012dbf1800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00822596 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0070159 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00915289 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00174813 s, 600 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00865878 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0088669 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848896 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129642 s, 80.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108143 s, 97.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00192405 s, 545 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0097262 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0013404 s, 782 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0017551 s, 597 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010388 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117305 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00313914 s, 334 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg354-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.140775 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126692 s, 82.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152227 s, 68.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138996 s, 75.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205267 s, 51.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342818 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266428 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034598 s, 30.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125829 s, 83.3 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156028 s, 67.2 MB/s , 0.0329672 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294479 s, 35.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012dbf1800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291549 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282119 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400287 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00938969 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00758787 s, 138 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00930148 s, 113 MB/s , 0.00963744 s, 109 MB/s 1048576 bytes (1.0 MB) copied, 0.00848049 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00822455 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118882 s, 88.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00950178 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0104414 s, 100 MB/s , 0.0140705 s, 74.5 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00980558 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00992279 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104889 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135252 s, 77.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124179 s, 84.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117103 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112212 s, 93.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107394 s, 97.6 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg354-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.100107 s, 168 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103056 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170716 s, 61.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130086 s, 80.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266088 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130692 s, 80.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0285454 s, 36.7 MB/s , 0.0334758 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0223166 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294931 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300148 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0303248 s, 34.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012dbf1800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342686 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0401539 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392866 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0504696 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0490586 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00976133 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00163299 s, 642 MB/s , 0.00195115 s, 537 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00974161 s, 108 MB/s , 0.00816743 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00122221 s, 858 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00864541 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0099924 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102896 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00660969 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105717 s, 99.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00629685 s, 167 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0187587 s, 55.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0176686 s, 59.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207618 s, 50.5 MB/s , 0.0184175 s, 56.9 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=4 removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg354-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (5s) == sanityn test 77d: check TRR nrs policy ================ 16:33:02 (1748982782) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.118433 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00681966 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00904639 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117012 s, 89.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239979 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237926 s, 44.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217511 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0303372 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208316 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261732 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022641 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027587 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343252 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332794 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300981 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0426006 s, 24.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012dbf1800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0419139 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00755096 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848384 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105231 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111584 s, 94.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00879004 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00881065 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104497 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110582 s, 94.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0108588 s, 96.6 MB/s , 0.0101226 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112161 s, 93.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0102809 s, 102 MB/s , 0.0109479 s, 95.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00973542 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114365 s, 91.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110619 s, 94.8 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg354-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.112101 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984818 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116563 s, 90.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254604 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0173758 s, 60.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258362 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292903 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270098 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273937 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331568 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0453176 s, 23.1 MB/s , 0.0266267 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0456461 s, 23.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012dbf1800.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0488919 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0480366 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0482933 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0437646 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00695298 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00683778 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00826129 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00922229 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112191 s, 93.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0096084 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00927923 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108354 s, 96.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00136048 s, 771 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00134158 s, 782 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00326686 s, 321 MB/s 1048576 bytes (1.0 MB) copied, 0.00388123 s, 270 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110611 s, 94.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00155848 s, 673 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00127761 s, 821 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00131706 s, 796 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg354-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0934498 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.035185 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0387924 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.035211 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129302 s, 81.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188237 s, 55.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0185324 s, 56.6 MB/s , 0.0443542 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145868 s, 71.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021588 s, 48.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248433 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0518757 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018115 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0659216 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0678261 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0631075 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134238 s, 78.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00828147 s, 127 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00519556 s, 202 MB/s 1048576 bytes (1.0 MB) copied, 0.00528713 s, 198 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00538835 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0019938 s, 526 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0073499 s, 143 MB/s , 0.00132445 s, 792 MB/s 1048576 bytes (1.0 MB) copied, 0.00574592 s, 182 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00964268 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00922082 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00929556 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118437 s, 88.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111928 s, 93.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00132299 s, 793 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141388 s, 74.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103589 s, 101 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg354-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (3s) == sanityn test 77e: check TBF NID nrs policy ============ 16:33:05 (1748982785) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.203.54@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.16661 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108718 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110254 s, 95.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250508 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331583 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277547 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213219 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243515 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0301363 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263961 s, 39.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197877 s, 53.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356902 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263964 s, 39.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0314254 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216564 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281792 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228283 s, 45.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012766 s, 82.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126522 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134358 s, 78.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013564 s, 77.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174222 s, 60.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0136466 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169332 s, 61.9 MB/s , 0.0129518 s, 81.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138578 s, 75.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148255 s, 70.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151476 s, 69.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142421 s, 73.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149954 s, 69.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154111 s, 68.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160096 s, 65.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178776 s, 58.7 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg354-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.143143 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00884194 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122509 s, 85.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128861 s, 81.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185389 s, 56.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197519 s, 53.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127038 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149926 s, 69.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102621 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00735269 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0337776 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.033841 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208352 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276416 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164845 s, 63.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187693 s, 55.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241314 s, 43.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0119186 s, 88.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0117437 s, 89.3 MB/s , 0.011916 s, 88.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161784 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114656 s, 91.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161191 s, 65.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121397 s, 86.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126699 s, 82.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143958 s, 72.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146496 s, 71.6 MB/s , 0.012975 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00262067 s, 400 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155656 s, 67.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148895 s, 70.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160396 s, 65.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180053 s, 58.2 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg354-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.116216 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103822 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158432 s, 66.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226403 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229923 s, 45.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328528 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248328 s, 42.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026876 s, 39.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0258159 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0318167 s, 33.0 MB/s 1048576 bytes (1.0 MB) copied, 0.027884 s, 37.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284975 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269663 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023908 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245144 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0228732 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319682 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00244721 s, 428 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010018 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00930124 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00958811 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118641 s, 88.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00531468 s, 197 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010794 s, 97.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105691 s, 99.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111895 s, 93.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110149 s, 95.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122769 s, 85.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984075 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106264 s, 98.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105233 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105962 s, 99.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119735 s, 87.6 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg354-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.117823 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116007 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111354 s, 94.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0215659 s, 48.6 MB/s , 0.00886402 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250557 s, 41.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231021 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240707 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386779 s, 27.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0339645 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0299065 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0322312 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365279 s, 28.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0394409 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398498 s, 26.3 MB/s , 0.0462066 s, 22.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012dbf1800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413458 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0045284 s, 232 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00441773 s, 237 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00369421 s, 284 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00486059 s, 216 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983799 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.005721 s, 183 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00501006 s, 209 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00558539 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00616519 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00576492 s, 182 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00164981 s, 636 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0029483 s, 356 MB/s 1048576 bytes (1.0 MB) copied, 0.00210556 s, 498 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00131501 s, 797 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00119003 s, 881 MB/s 1048576 bytes (1.0 MB) copied, 0.00342334 s, 306 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=1 removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg354-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (8s) == sanityn test 77f: check TBF JobID nrs policy ========== 16:33:13 (1748982793) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg354-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.103523 s, 162 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143172 s, 73.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138176 s, 75.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133031 s, 78.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0185329 s, 56.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013748 s, 76.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175604 s, 59.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184051 s, 57.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0366704 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0377676 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163543 s, 64.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186802 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0311223 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132162 s, 79.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242547 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00828367 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00940727 s, 111 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00890147 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00756795 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0081696 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012859 s, 81.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129092 s, 81.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133919 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136641 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143245 s, 73.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116531 s, 90.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163156 s, 64.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195143 s, 53.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123869 s, 84.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121004 s, 86.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121487 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138467 s, 75.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00395237 s, 265 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg354-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg354-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.113459 s, 148 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00889205 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116528 s, 90.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146661 s, 71.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00810288 s, 129 MB/s 1048576 bytes (1.0 MB) copied, 0.0236903 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017606 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236812 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190979 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275978 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180037 s, 58.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0252684 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0389098 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015185 s, 69.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00952209 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293356 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158177 s, 66.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122155 s, 85.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116603 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010672 s, 98.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00987404 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116523 s, 90.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00148938 s, 704 MB/s , 0.00885733 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01142 s, 91.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101813 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00993495 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00999497 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00844794 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104458 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113391 s, 92.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117745 s, 89.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130264 s, 80.5 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg354-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg354-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.128487 s, 131 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011474 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145146 s, 72.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149376 s, 70.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120115 s, 87.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152849 s, 68.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365117 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317436 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341271 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272569 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261874 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245703 s, 42.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.043521 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0451697 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0426727 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354233 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0493429 s, 21.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012dbf1800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=2 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00938042 s, 112 MB/s , 0.00164143 s, 639 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00859825 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00865748 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01068 s, 98.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00766849 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0093893 s, 112 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00837943 s, 125 MB/s , 0.00868655 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00643037 s, 163 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0032613 s, 322 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00814283 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00206351 s, 508 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00238979 s, 439 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00137006 s, 765 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00142717 s, 735 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg354-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg354-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0950109 s, 177 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114721 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134015 s, 78.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155757 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241953 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219957 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200963 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0334807 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0347798 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267211 s, 39.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237067 s, 44.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0410304 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203459 s, 51.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300442 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232603 s, 45.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0301675 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216757 s, 48.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012dbf1800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=1 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00679378 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00907996 s, 115 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968824 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00856838 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00391756 s, 268 MB/s 1+0 records in 1+0 records out , 0.00675722 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00147092 s, 713 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.010622 s, 98.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0106279 s, 98.7 MB/s 1048576 bytes (1.0 MB) copied, 0.00481531 s, 218 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00363379 s, 289 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126522 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130896 s, 80.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121662 s, 86.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00213754 s, 491 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012935 s, 81.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=1 removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg354-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (7s) == sanityn test 77g: Change TBF type directly ============ 16:33:20 (1748982800) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (4s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 16:33:24 (1748982804) oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg354-client: oleg354-server: ssh exited with exit code 2 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg354-client: oleg354-server: ssh exited with exit code 12 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg354-client: oleg354-server: ssh exited with exit code 12 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg354-client: oleg354-server: ssh exited with exit code 12 PASS 77h (3s) == sanityn test 77i: Change rank of TBF rule ============= 16:33:27 (1748982807) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (6s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 16:33:33 (1748982813) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0857698 s, 196 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00815861 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00905937 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00730986 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141367 s, 74.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211173 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178483 s, 58.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295302 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292966 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207925 s, 50.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183221 s, 57.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258585 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356345 s, 29.4 MB/s , 0.0230303 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364794 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273093 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284469 s, 36.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0108985 s, 96.2 MB/s , 0.0141289 s, 74.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012327 s, 85.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131166 s, 79.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136797 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161039 s, 65.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0152446 s, 68.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156351 s, 67.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0148679 s, 70.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0149535 s, 70.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161705 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152065 s, 69.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181103 s, 57.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0176154 s, 59.5 MB/s , 0.0172459 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212848 s, 49.3 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg354-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94548 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4374 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (50s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 16:34:23 (1748982863) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg354-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.441338 s, 38.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00915082 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117691 s, 89.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127601 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149048 s, 70.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00896084 s, 117 MB/s 1048576 bytes (1.0 MB) copied, 0.00730885 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00686812 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134418 s, 78.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.373141 s, 2.8 MB/s , 0.3711 s, 2.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.375697 s, 2.8 MB/s , 0.383405 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.387121 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.386589 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.398415 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.383425 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145351 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146854 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147708 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.146215 s, 7.2 MB/s 1048576 bytes (1.0 MB) copied, 0.146359 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147668 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14715 s, 7.1 MB/s 1048576 bytes (1.0 MB) copied, 0.146949 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148628 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147174 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147696 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146912 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147265 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148548 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148991 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148859 s, 7.0 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg354-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8326 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.969 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg354-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.656591 s, 25.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145314 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.170474 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159178 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157134 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.177028 s, 5.9 MB/s 1048576 bytes (1.0 MB) copied, 0.175187 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.169178 s, 6.2 MB/s 1048576 bytes (1.0 MB) copied, 0.166174 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.335113 s, 3.1 MB/s 1048576 bytes (1.0 MB) copied, 0.346549 s, 3.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.357495 s, 2.9 MB/s 1+0 records in 1+0 records out , 0.355559 s, 2.9 MB/s 1048576 bytes (1.0 MB) copied, 0.3377 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.364929 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.356747 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.374403 s, 2.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.117988 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.12436 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.118409 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.12338 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121059 s, 8.7 MB/s 1048576 bytes (1.0 MB) copied, 0.12372 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.117991 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.126299 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.120665 s, 8.7 MB/s , 0.119839 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.120342 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.119475 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.12054 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.12475 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.12449 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.125952 s, 8.3 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg354-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.7135 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9351 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (132s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 16:36:35 (1748982995) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.190285 s, 88.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164566 s, 63.7 MB/s , 0.0257491 s, 40.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0301304 s, 34.8 MB/s , 0.0353808 s, 29.6 MB/s , 0.0305601 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285244 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0329418 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.041058 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0751305 s, 14.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0840457 s, 12.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0758019 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0843999 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0962329 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.080513 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0997274 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0858565 s, 12.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00938387 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00899184 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00839085 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00859006 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0086429 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107186 s, 97.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108225 s, 96.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130275 s, 80.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00140736 s, 745 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012166 s, 86.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119932 s, 87.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122072 s, 85.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105806 s, 99.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141717 s, 74.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110389 s, 95.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138974 s, 75.5 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.9355 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73606 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.54@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.54@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.16468 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192326 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128624 s, 81.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0401913 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272102 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0427924 s, 24.5 MB/s , 0.0410314 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0420881 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0348942 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0299995 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319407 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.129018 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14395 s, 7.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.231396 s, 4.5 MB/s , 0.240506 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.325952 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.332341 s, 3.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153357 s, 68.4 MB/s , 0.0125933 s, 83.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187175 s, 56.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0149624 s, 70.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156492 s, 67.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0157714 s, 66.5 MB/s , 0.0160612 s, 65.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169172 s, 62.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158797 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162468 s, 64.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179255 s, 58.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170117 s, 61.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174585 s, 60.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179925 s, 58.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185753 s, 56.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203336 s, 51.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96722 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74515 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.54@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.181012 s, 92.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201877 s, 51.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0329853 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248662 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0433508 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365733 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388054 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398074 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.04602 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0621809 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0547398 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0641225 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0681045 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0657231 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0722202 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0772052 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0798472 s, 13.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107555 s, 97.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136116 s, 77.0 MB/s , 0.0138675 s, 75.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126134 s, 83.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154508 s, 67.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176653 s, 59.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199712 s, 52.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122572 s, 85.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142772 s, 73.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144952 s, 72.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136662 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159405 s, 65.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00364626 s, 288 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171007 s, 61.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160609 s, 65.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189117 s, 55.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg354-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96211 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.89548 s, 21.4 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.277699 s, 60.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203787 s, 51.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0348069 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327024 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0404917 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03004 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.032699 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254189 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324813 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.136008 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.120424 s, 8.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.321833 s, 3.3 MB/s 1048576 bytes (1.0 MB) copied, 0.313913 s, 3.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.340671 s, 3.1 MB/s , 0.343766 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.387395 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.374366 s, 2.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133966 s, 78.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129543 s, 80.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112217 s, 93.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103409 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110842 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127107 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128059 s, 81.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132812 s, 79.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140441 s, 74.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147832 s, 70.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157156 s, 66.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141065 s, 74.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132361 s, 79.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0172895 s, 60.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0177693 s, 59.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162833 s, 64.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.79969 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73713 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96643 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74824 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.477746 s, 35.1 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161762 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198966 s, 52.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341737 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233547 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304236 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0493027 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0410601 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0467976 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.335938 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.355102 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.347675 s, 3.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.350722 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.363377 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.347435 s, 3.0 MB/s 1048576 bytes (1.0 MB) copied, 0.355247 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.352687 s, 3.0 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132399 s, 79.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102291 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104057 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109373 s, 95.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0115383 s, 90.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182182 s, 57.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132691 s, 79.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144662 s, 72.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190479 s, 55.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145847 s, 71.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159958 s, 65.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.015343 s, 68.3 MB/s , 0.0149792 s, 70.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166556 s, 63.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171517 s, 61.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187224 s, 56.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.7329 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4447 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.210003 s, 79.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114545 s, 91.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013419 s, 78.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0305688 s, 34.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0296098 s, 35.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0265202 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0329922 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261776 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274951 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0557539 s, 18.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0706254 s, 14.8 MB/s , 0.0715113 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0638418 s, 16.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0710697 s, 14.8 MB/s , 0.0739193 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0854557 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0786855 s, 13.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115149 s, 91.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0107386 s, 97.6 MB/s , 0.0113059 s, 92.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011462 s, 91.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141127 s, 74.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175848 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168947 s, 62.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00308873 s, 339 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171082 s, 61.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0231714 s, 45.3 MB/s , 0.0168105 s, 62.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159747 s, 65.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159178 s, 65.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155155 s, 67.6 MB/s , 0.0155057 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174465 s, 60.1 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg354-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96196 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74766 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.263027 s, 63.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142761 s, 73.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294967 s, 35.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0319064 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217206 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207591 s, 50.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0303527 s, 34.5 MB/s , 0.0356305 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282544 s, 37.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.125853 s, 8.3 MB/s 1048576 bytes (1.0 MB) copied, 0.137437 s, 7.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.217339 s, 4.8 MB/s , 0.239421 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.290879 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.288728 s, 3.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.329688 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.319093 s, 3.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127543 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166508 s, 63.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147229 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130589 s, 80.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147345 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150028 s, 69.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162455 s, 64.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169518 s, 61.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163233 s, 64.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141506 s, 74.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146111 s, 71.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158435 s, 66.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154934 s, 67.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016069 s, 65.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174395 s, 60.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169602 s, 61.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg354-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.81414 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74274 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.8093 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74435 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (357s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 16:42:32 (1748983352) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (3s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 16:42:36 (1748983356) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg354-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.94294 s, 266 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg354-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.026 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (52s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 16:43:27 (1748983407) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.160438 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123005 s, 85.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353137 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0410222 s, 25.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386514 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218714 s, 47.9 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019158 s, 54.7 MB/s , 0.0410538 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270687 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0414695 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0399761 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0445688 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0350738 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0285959 s, 36.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0353549 s, 29.7 MB/s , 0.0273224 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025497 s, 41.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012dbf1800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88012dbf1800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0088099 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00945205 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00909288 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00952619 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112352 s, 93.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107639 s, 97.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985694 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00547763 s, 191 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115552 s, 90.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100786 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107259 s, 97.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137792 s, 76.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127515 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124452 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123979 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141237 s, 74.2 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg354-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95368 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97485 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.188065 s, 89.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129865 s, 80.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119913 s, 87.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0082578 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013121 s, 79.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00996044 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171055 s, 61.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00881835 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119773 s, 87.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.072349 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0862663 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.084102 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0856888 s, 12.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.159349 s, 6.6 MB/s , 0.172125 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.26436 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.273997 s, 3.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149723 s, 70.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0124104 s, 84.5 MB/s , 0.0170149 s, 61.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129655 s, 80.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155449 s, 67.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00817672 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162656 s, 64.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201084 s, 52.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177293 s, 59.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168414 s, 62.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196977 s, 53.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187358 s, 56.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196892 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019994 s, 52.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192341 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0223112 s, 47.0 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg354-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg354-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95853 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96866 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (65s) == sanityn test 77o: Changing rank should not panic ====== 16:44:32 (1748983472) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (3s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 16:44:35 (1748983475) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (31s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 16:45:06 (1748983506) mds.MDS.mdt.nrs_policies=tbf oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg354-client: oleg354-server: ssh exited with exit code 36 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg354-client: oleg354-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (12s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 16:45:18 (1748983518) oleg354-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: Resource temporarily unavailable pdsh@oleg354-client: oleg354-server: ssh exited with exit code 11 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (3s) == sanityn test 79: xattr: intent error ================== 16:45:21 (1748983521) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505431 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-06-03 16:45:22.000000000 -0400 Modify: 2025-06-03 16:45:22.000000000 -0400 Change: 2025-06-03 16:45:22.000000000 -0400 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (13s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 16:45:34 (1748983534) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 16:45:35 (1748983535) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 16:45:37 (1748983537) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 16:45:38 (1748983538) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 16:45:39 (1748983539) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 16:45:40 (1748983540) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6866 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6866 PASS 82 (2s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 16:45:42 (1748983542) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 16:45:43 (1748983543) fail_loc=0x8000060b PASS 84 (7s) == sanityn test 90: open/create and unlink striped directory ========================================================== 16:45:50 (1748983550) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 16:45:51 (1748983551) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 16:45:53 (1748983553) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 16:45:54 (1748983554) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2642 0xa52 0 1 2629 0xa45 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2643 0xa53 0 1 2630 0xa46 0 rc1=0 and rc2=0 PASS 93 (9s) == sanityn test 94: signal vs CP callback race =========== 16:46:03 (1748983563) osc.lustre-OST0000-osc-ffff88012bf96000.idle_timeout=0 osc.lustre-OST0000-osc-ffff88012dbf1800.idle_timeout=0 osc.lustre-OST0001-osc-ffff88012bf96000.idle_timeout=0 osc.lustre-OST0001-osc-ffff88012dbf1800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0504055 s, 163 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 2.99882 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 4500 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0231668 s, 177 kB/s osc.lustre-OST0000-osc-ffff88012bf96000.idle_timeout=debug osc.lustre-OST0000-osc-ffff88012dbf1800.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012bf96000.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012dbf1800.idle_timeout=debug PASS 94 (10s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 16:46:13 (1748983573) llite.lustre-ffff88012bf96000.fast_read=0 llite.lustre-ffff88012dbf1800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00270139 s, 4.5 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff88012bf96000.fast_read=1 llite.lustre-ffff88012dbf1800.fast_read=1 PASS 95b (9s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 16:46:22 (1748983582) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 16:46:23 (1748983583) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00147174 s, 2.8 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (1s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 16:46:24 (1748983584) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0199468 s, 105 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 16:46:26 (1748983586) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.02 s, 105 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (1s) == sanityn test 100e: DoM: read on open and file size ==== 16:46:27 (1748983587) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00148091 s, 8.1 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 16:46:29 (1748983589) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000432379 s, 9.5 MB/s PASS 101a (1s) == sanityn test 101b: Discard DoM data on rename ========= 16:46:30 (1748983590) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000403875 s, 10.1 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 16:46:32 (1748983592) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000555242 s, 7.4 MB/s PASS 101c (2s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 16:46:35 (1748983595) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1748983596 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1748983598 Finishing test test23 at 1748983598 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 16:46:42 (1748983602) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 16:46:43 (1748983603) fail_loc=0x416 PASS 105 (47s) == sanityn test 106a: Verify the btime via statx() ======= 16:47:31 (1748983651) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 16:47:32 (1748983652) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) == sanityn test 106c: Verify statx attributes mask ======= 16:47:33 (1748983653) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) == sanityn test 107a: Basic grouplock conflict =========== 16:47:34 (1748983654) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.082024 s, 128 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2650 0xa5a 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6866 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6866 First grouplock blocks second one PASS 107a (5s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 16:47:39 (1748983659) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0844448 s, 124 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2634 0xa4a 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6866 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) == sanityn test 108a: lseek: parallel updates ============ 16:47:47 (1748983667) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00287136 s, 3.5 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.28209 s, 2.0 MB/s PASS 108a (7s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 16:47:54 (1748983674) 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions oleg354-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 6 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions oleg354-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 11 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions oleg354-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 12 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions oleg354-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 30 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions oleg354-server: ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' oleg354-server: insmod: ERROR: could not insert module /home/green/git/lustre-release/lustre/../libcfs/libcfs/libcfs.ko: File exists pdsh@oleg354-client: oleg354-server: ssh exited with exit code 1 Iteration 32 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions oleg354-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 33 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions oleg354-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg354-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 35 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions oleg354-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 38 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions oleg354-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 48 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions oleg354-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 49 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 192.168.203.154@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.154@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg354-client.virtnet /mnt/lustre (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg354-server' oleg354-server: oleg354-server.virtnet: executing load_modules_local oleg354-server: Loading modules from /home/green/git/lustre-release/lustre oleg354-server: detected 4 online CPUs by sysfs oleg354-server: Force libcfs to create 2 CPU partitions oleg354-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre Starting client: oleg354-client.virtnet: -o user_xattr,flock oleg354-server@tcp:/lustre /mnt/lustre2 PASS 109 (455s) == sanityn test 110: do not grant another lock on resend ========================================================== 16:55:29 (1748984129) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-06-03 16:55:30.000000000 -0400 Modify: 2025-06-03 16:55:30.000000000 -0400 Change: 2025-06-03 16:55:30.000000000 -0400 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg354-client.virtnet oleg354-client.virtnet PASS 110 (37s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 16:56:06 (1748984166) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 16:56:07 (1748984167) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 16:56:08 (1748984168) lustre-OST0000-osc-ffff8800b5c2b000 active. lustre-OST0001-osc-ffff8800b5c2b000 active. lustre-OST0000-osc-ffff8800b5c2b000 active. lustre-OST0001-osc-ffff8800b5c2b000 active. MGC192.168.203.154@tcp active. PASS 113 (3s) == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 16:56:11 (1748984171) SKIP: sanityn test_115 ldiskfs only test SKIP 115 (1s) cleanup: ====================================================== == sanityn test complete, duration 4026 sec ============== 16:56:13 (1748984173) Stopping clients: oleg354-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg354-client.virtnet /mnt/lustre2 opts: