-----============= acceptance-small: sanityn ============----- Wed Jan 8 12:35:34 EST 2025 excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 Started clients oleg324-client.virtnet: 192.168.203.124@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt) oleg324-client.virtnet: executing check_config_client /mnt/lustre oleg324-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg324-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff88012a60b800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8801369f4000.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012a60b800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8801369f4000.idle_timeout=debug disable quota as required oleg324-server: oleg324-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106396 s, 98.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6858] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 12:35:44 (1736357744) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (1s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 12:35:45 (1736357745) -rw-r--r-- 1 root root 0 Jan 8 12:35 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (2s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 12:35:47 (1736357747) -rw-r--r-- 1 root root 0 Jan 8 12:35 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (1s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 12:35:48 (1736357748) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (2s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 12:35:50 (1736357750) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (1s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 12:35:51 (1736357751) total 1 -rwxrwxrwx 1 root root 0 Jan 8 12:35 f2a -rwxrwxrwx 1 root root 0 Jan 8 12:35 f2b total 1 -rwxrwxrwx 1 root root 0 Jan 8 12:35 f2a -rwxrwxrwx 1 root root 0 Jan 8 12:35 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00231787 s, 221 kB/s PASS 2e (1s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 12:35:53 (1736357753) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (0s) == sanityn test 2g: check blocks update on sync write ==== 12:35:53 (1736357753) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0720078 s, 29.1 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (2s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 12:35:55 (1736357755) PASS 3 (1s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 12:35:56 (1736357756) PASS 4 (3s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 12:35:59 (1736357759) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (1s) == sanityn test 6: remove of open file on other node ============================================================================ 12:36:00 (1736357760) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (2s) == sanityn test 7: remove of open directory on other node ======================================================================= 12:36:02 (1736357762) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (2s) == sanityn test 8: remove of open special file on other node ==================================================================== 12:36:04 (1736357764) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (2s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 12:36:06 (1736357766) PASS 9a (2s) == sanityn test 9b: append to striped sparse file ======== 12:36:08 (1736357768) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172531 s, 60.8 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0151109 s, 0.2 kB/s Data read (expecting 'foo'): foo PASS 9b (2s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 12:36:10 (1736357770) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00181668 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00139386 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00173666 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00123435 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00192119 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00148015 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00140498 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00114536 s, 0.9 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00148165 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00170849 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00119005 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00129405 s, 0.8 kB/s PASS 10a (2s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 12:36:12 (1736357772) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00227301 s, 1.4 MB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00594552 s, 689 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00022086 s, 13.9 MB/s PASS 10b (1s) == sanityn test 11: execution of file opened for write should return error ============================================================== 12:36:13 (1736357773) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6858 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (2s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 12:36:15 (1736357775) warning: '-runtime' deprecated, use '-t runtime' instead - link 2622 (time 1736357785.80 total 10.00 last 262.12) - link 5119 (time 1736357795.80 total 20.00 last 249.67) - link 6997 (time 1736357805.80 total 30.00 last 187.80) - link 9840 (time 1736357815.81 total 40.01 last 284.24) - link 13365 (time 1736357825.81 total 50.01 last 352.42) - link 15700 (time 1736357835.81 total 60.01 last 233.49) - link 17807 (time 1736357845.81 total 70.01 last 210.64) - link 20000 (time 1736357853.90 total 78.10 last 271.16) - link 22414 (time 1736357863.90 total 88.10 last 241.38) - link 24424 (time 1736357873.90 total 98.10 last 200.96) total: 24817 link in 100.00 seconds: 248.16 ops/second using seed 1633550427 running for 100 seconds - stat 10000 (time 1736357779 ; total 3 ; last 3) - stat 20000 (time 1736357782 ; total 6 ; last 3) - stat 30000 (time 1736357785 ; total 9 ; last 3) - stat 40000 (time 1736357788 ; total 12 ; last 3) - stat 50000 (time 1736357791 ; total 15 ; last 3) - stat 60000 (time 1736357795 ; total 19 ; last 4) - stat 70000 (time 1736357798 ; total 22 ; last 3) - stat 80000 (time 1736357801 ; total 25 ; last 3) - stat 90000 (time 1736357804 ; total 28 ; last 3) - stat 100000 (time 1736357807 ; total 31 ; last 3) - stat 110000 (time 1736357810 ; total 34 ; last 3) - stat 120000 (time 1736357813 ; total 37 ; last 3) - stat 130000 (time 1736357816 ; total 40 ; last 3) - stat 140000 (time 1736357819 ; total 43 ; last 3) - stat 150000 (time 1736357821 ; total 45 ; last 2) - stat 160000 (time 1736357824 ; total 48 ; last 3) - stat 170000 (time 1736357827 ; total 51 ; last 3) - stat 180000 (time 1736357834 ; total 58 ; last 7) - stat 190000 (time 1736357842 ; total 66 ; last 8) - stat 200000 (time 1736357848 ; total 72 ; last 6) - stat 210000 (time 1736357854 ; total 78 ; last 6) - stat 220000 (time 1736357860 ; total 84 ; last 6) - stat 230000 (time 1736357868 ; total 92 ; last 8) - stat 240000 (time 1736357875 ; total 99 ; last 7) total: 240987 stats in 100 seconds: 2409.870117 stats/second - unlinked 0 (time 1736357826 ; total 0 ; last 0) - unlinked 10000 (time 1736357884 ; total 58 ; last 58) - unlinked 20000 (time 1736357908 ; total 82 ; last 24) unlink(/mnt/lustre2/lockdir/lockfile24817) error: No such file or directory total: 24816 unlinks in 94 seconds: 264.000000 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (20453) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (20456) - No such process PASS 12 (146s) == sanityn test 13: test directory page revocation ======= 12:38:41 (1736357921) f13 f13 PASS 13 (2s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 12:38:43 (1736357923) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6858 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4260: 21835 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (1s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 12:38:44 (1736357924) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4260: 22463 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (1s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 12:38:45 (1736357925) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 23089 Terminated $DIR1/$tdir/sleep 60 PASS 14b (2s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 12:38:47 (1736357927) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 23718 Terminated $DIR1/$tdir/sleep 60 PASS 14c (1s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 12:38:48 (1736357928) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 24346 Terminated $DIR1/$tdir/sleep 60 PASS 14d (2s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 12:38:50 (1736357930) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7517184 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7517184kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (10s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 12:39:00 (1736357940) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.073625 s, 28.5 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0645706 s, 32.5 MB/s Chance of close/open is 1 in 50 Seed set to 7941 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read truncating to largest ever: 0x51e59a 000005[1] 1736357941.646148 trunc from 00000000 to 0x51e599 (0x51e59a bytes) 000006[0] 1736357941.660706 read 0x3da2fd thru 0x3e2ac4 (0x87c8 bytes) 000008[1] 1736357941.670455 trunc from 0x51e59a to 0x4bd9ac (0x60bed bytes) 000011[0] 1736357941.685146 write 0x2e00ae thru 0x2ef2dc (0xf22f bytes) 000013[0] 1736357941.696423 mapwrite 0x54f2bb thru 0x554bbe (0x5904 bytes) 000014[0] 1736357941.735903 read 0x1d82e7 thru 0x1e19af (0x96c9 bytes) 000016[0] 1736357941.757189 mapwrite 0x32216b thru 0x32b105 (0x8f9b bytes) 000018[0] 1736357941.773670 trunc from 0x554bbf to 0x34505b (0x20fb63 bytes) 000019[1] 1736357941.782521 mapread 0x1315b4 thru 0x13b998 (0xa3e5 bytes) 000020[1] 1736357941.792812 mapwrite 0x2ca3ab thru 0x2d4cf3 (0xa949 bytes) 000021[0] 1736357941.821323 mapwrite 0x224d27 thru 0x227fb1 (0x328b bytes) 000023[0] 1736357941.850195 write 0x7b4ac6 thru 0x7bac90 (0x61cb bytes) truncating to largest ever: 0x9f7c30 000025[1] 1736357941.857411 trunc from 0x7bac91 to 0x9f7c2f (0x23cf9f bytes) 000029[0] 1736357941.871716 mapread 0x86cece thru 0x86ef04 (0x2037 bytes) 000035[1] 1736357941.885766 mapread 0x091977 thru 0x0a179d (0xfe27 bytes) 000037[1] 1736357941.895581 mapread 0x770654 thru 0x774666 (0x4013 bytes) 000041[0] 1736357941.905003 mapwrite 0x6cda50 thru 0x6d1627 (0x3bd8 bytes) 000042[1] 1736357941.933129 mapwrite 0x23ec25 thru 0x248d03 (0xa0df bytes) 000044[1] 1736357941.955163 write 0x6150ca thru 0x62011d (0xb054 bytes) 000046[1] 1736357941.958108 read 0x49d7b1 thru 0x4ad025 (0xf875 bytes) 000047[0] 1736357941.963851 read 0x408cdb thru 0x4163bd (0xd6e3 bytes) 000048[1] 1736357941.976454 mapwrite 0x151c78 thru 0x151e46 (0x1cf bytes) 000049[1] 1736357941.994907 mapwrite 0x42cdfc thru 0x42d99f (0xba4 bytes) 000053[0] 1736357942.016572 read 0x2ddce4 thru 0x2e927a (0xb597 bytes) 000054[1] 1736357942.022944 mapwrite 0x3ace79 thru 0x3b68e0 (0x9a68 bytes) 000056[0] 1736357942.042573 mapwrite 0x666ea7 thru 0x66811b (0x1275 bytes) 000059[1] 1736357942.077935 read 0x61505a thru 0x624ce4 (0xfc8b bytes) 000060[1] 1736357942.099020 mapwrite 0x662de5 thru 0x66fc57 (0xce73 bytes) 000062[0] 1736357942.145424 mapread 0x07ceca thru 0x082b67 (0x5c9e bytes) 000064[1] 1736357942.161640 write 0x049db3 thru 0x0551ba (0xb408 bytes) 000065[1] 1736357942.170207 mapwrite 0x43e443 thru 0x4461d5 (0x7d93 bytes) 000069[0] 1736357942.197574 trunc from 0x9f7c30 to 0x5bf776 (0x4384b9 bytes) 000073[1] 1736357942.215125 mapwrite 0x7c16bb thru 0x7c8183 (0x6ac9 bytes) 000074[1] 1736357942.240271 read 0x6bda6b thru 0x6c9fae (0xc544 bytes) 000075[0] 1736357942.245017 mapwrite 0x0c91ed thru 0x0cf18f (0x5fa3 bytes) 000076[1] 1736357942.274922 write 0x0ff400 thru 0x109730 (0xa331 bytes) 000079[1] 1736357942.284901 write 0x19ac11 thru 0x1a8f9b (0xe38b bytes) 000080[1] 1736357942.288304 mapwrite 0x83fe63 thru 0x849f27 (0xa0c5 bytes) 000081[0] 1736357942.313920 trunc from 0x849f28 to 0x8a9c9f (0x5fd78 bytes) 000082[0] 1736357942.323823 mapread 0x1e210a thru 0x1eb1fc (0x90f3 bytes) 000083[0] 1736357942.358745 write 0x046c13 thru 0x053781 (0xcb6f bytes) 000086[1] 1736357942.368353 mapread 0x0a7154 thru 0x0aafce (0x3e7b bytes) 000087[0] 1736357942.381570 read 0x7e4031 thru 0x7eecf6 (0xacc6 bytes) 000090[0] 1736357942.389706 write 0x9ec704 thru 0x9f6df3 (0xa6f0 bytes) 000092[0] 1736357942.394325 read 0x2b1fa9 thru 0x2b68a1 (0x48f9 bytes) 000093[0] 1736357942.408922 trunc from 0x9f6df4 to 0x941c35 (0xb51be bytes) 000095[0] 1736357942.424215 trunc from 0x941c36 to 0x7ff3eb (0x14284a bytes) 000096[1] 1736357942.431442 mapwrite 0x930ecb thru 0x932373 (0x14a9 bytes) 000097[0] 1736357942.456811 mapwrite 0x3268ff thru 0x32a1b7 (0x38b9 bytes) 000099[1] 1736357942.483614 mapwrite 0x155e32 thru 0x16206a (0xc239 bytes) 000101[0] 1736357942.519770 trunc from 0x932374 to 0x6d8c91 (0x2596e2 bytes) 000102[1] 1736357942.528283 trunc from 0x6d8c92 to 0x2dff80 (0x3f8d11 bytes) 000103[0] 1736357942.536103 write 0x792b9a thru 0x79963f (0x6aa6 bytes) 000104[1] 1736357942.543282 write 0x8be40d thru 0x8cd67f (0xf273 bytes) 000106[0] 1736357942.550063 trunc from 0x8cd680 to 0x17ebb7 (0x74eac8 bytes) 000111[1] 1736357942.558895 trunc from 0x17ebb8 to 0x35493c (0x1d5d85 bytes) 000112[0] 1736357942.566032 trunc from 0x35493d to 0x30fa61 (0x44edb bytes) 000114[1] 1736357942.577451 write 0x6d609c thru 0x6dfe0f (0x9d74 bytes) 000115[1] 1736357942.583602 write 0x07e10f thru 0x080ff8 (0x2eea bytes) 000121[1] 1736357942.604004 write 0x56abba thru 0x579759 (0xeba0 bytes) 000122[0] 1736357942.608389 mapread 0x02a10e thru 0x031971 (0x7864 bytes) 000124[0] 1736357942.628236 read 0x4a20bc thru 0x4b024c (0xe191 bytes) 000125[1] 1736357942.635112 trunc from 0x6dfe10 to 0x1f7955 (0x4e84ba bytes) 000126[1] 1736357942.642011 read 0x0d14b8 thru 0x0dae95 (0x99de bytes) 000127[1] 1736357942.651804 read 0x0fad8d thru 0x10ab16 (0xfd8a bytes) 000128[1] 1736357942.657926 mapread 0x0001af thru 0x000b90 (0x9e2 bytes) 000129[0] 1736357942.661807 write 0x59e490 thru 0x5a2d57 (0x48c8 bytes) 000130[0] 1736357942.668332 trunc from 0x5a2d58 to 0x07f724 (0x523633 bytes) 000131[1] 1736357942.677386 mapwrite 0x6049fc thru 0x610e1a (0xc41f bytes) 000132[1] 1736357942.712392 read 0x1a8236 thru 0x1b6cac (0xea77 bytes) 000133[1] 1736357942.729772 trunc from 0x610e1b to 0x2d19e3 (0x33f437 bytes) 000134[1] 1736357942.739562 read 0x1dc51e thru 0x1e5d90 (0x9873 bytes) 000135[0] 1736357942.740603 mapread 0x23fb30 thru 0x240478 (0x949 bytes) 000137[1] 1736357942.754016 mapwrite 0x13a8d5 thru 0x145cf9 (0xb425 bytes) 000138[0] 1736357942.770707 read 0x10c330 thru 0x11ada7 (0xea78 bytes) 000140[0] 1736357942.784768 write 0x07fa8c thru 0x0807c3 (0xd38 bytes) 000144[1] 1736357942.798186 mapread 0x1d8d78 thru 0x1e6d15 (0xdf9e bytes) 000148[1] 1736357942.820458 trunc from 0x2d19e4 to 0x368d3b (0x97358 bytes) 000149[1] 1736357942.834348 mapwrite 0x82eada thru 0x83e3fa (0xf921 bytes) 000151[1] 1736357942.880358 write 0x026563 thru 0x02ae47 (0x48e5 bytes) 000152[1] 1736357942.883900 mapread 0x1e4264 thru 0x1e767d (0x341a bytes) 000153[0] 1736357942.892632 mapread 0x76de17 thru 0x76e300 (0x4ea bytes) 000154[0] 1736357942.901457 trunc from 0x83e3fb to 0x978060 (0x139c66 bytes) 000155[1] 1736357942.915131 read 0x7ad8f9 thru 0x7ade31 (0x539 bytes) 000158[0] 1736357942.926463 read 0x093c5e thru 0x0a23ad (0xe750 bytes) 000159[1] 1736357942.953437 read 0x0414ec thru 0x044f2f (0x3a44 bytes) 000160[0] 1736357942.976470 mapwrite 0x28c27f thru 0x29a1e5 (0xdf67 bytes) 000164[0] 1736357943.017031 read 0x8d8ec7 thru 0x8db94b (0x2a85 bytes) 000166[0] 1736357943.023174 write 0x0f6c6a thru 0x0ffcf6 (0x908d bytes) 000169[0] 1736357943.026184 trunc from 0x978061 to 0x820814 (0x15784c bytes) 000172[0] 1736357943.045599 read 0x35cf15 thru 0x35e026 (0x1112 bytes) 000174[0] 1736357943.067455 read 0x389029 thru 0x38d774 (0x474c bytes) 000176[1] 1736357943.068045 trunc from 0x820815 to 0x91aa8a (0xfa276 bytes) 000177[1] 1736357943.084111 write 0x57ee1e thru 0x585f2f (0x7112 bytes) 000178[1] 1736357943.090095 mapwrite 0x1cd189 thru 0x1dcfe0 (0xfe58 bytes) 000180[0] 1736357943.122129 write 0x8e3dff thru 0x8ee80e (0xaa10 bytes) 000182[1] 1736357943.134552 trunc from 0x91aa8b to 0x52e21e (0x3ec86c bytes) 000183[0] 1736357943.148984 write 0x550ed7 thru 0x55379b (0x28c5 bytes) 000185[1] 1736357943.160875 mapwrite 0x46a948 thru 0x477dcc (0xd485 bytes) 000187[1] 1736357943.198656 mapread 0x452cda thru 0x45eea8 (0xc1cf bytes) 000190[0] 1736357943.236720 mapwrite 0x587c6d thru 0x595651 (0xd9e5 bytes) 000191[0] 1736357943.278439 read 0x373d70 thru 0x378161 (0x43f2 bytes) 000194[1] 1736357943.298220 write 0x3d1f64 thru 0x3e0723 (0xe7c0 bytes) 000196[1] 1736357943.311114 mapwrite 0x0cf2bc thru 0x0db278 (0xbfbd bytes) 000197[1] 1736357943.348624 mapwrite 0x497a52 thru 0x4a005f (0x860e bytes) 000201[1] 1736357943.382138 read 0x2a7147 thru 0x2abb17 (0x49d1 bytes) 000202[0] 1736357943.394037 read 0x4d1f0c thru 0x4d5105 (0x31fa bytes) 000203[1] 1736357943.403000 mapread 0x3bc7ef thru 0x3cb3a8 (0xebba bytes) 000204[1] 1736357943.404122 mapwrite 0x0ebf66 thru 0x0f1257 (0x52f2 bytes) 000206[0] 1736357943.423499 read 0x17f2f8 thru 0x18668b (0x7394 bytes) 000207[1] 1736357943.439015 mapwrite 0x720ecb thru 0x72de2c (0xcf62 bytes) 000208[0] 1736357943.486871 mapread 0x3b70f4 thru 0x3bb43b (0x4348 bytes) 000210[1] 1736357943.500356 read 0x4f7baf thru 0x5025c1 (0xaa13 bytes) 000212[0] 1736357943.512315 write 0x51319d thru 0x51cd4f (0x9bb3 bytes) 000215[1] 1736357943.526417 read 0x2a9ec6 thru 0x2abbbb (0x1cf6 bytes) 000217[0] 1736357943.535654 mapwrite 0x9b79a1 thru 0x9c07d5 (0x8e35 bytes) 000219[1] 1736357943.584632 mapread 0x7772c5 thru 0x783b26 (0xc862 bytes) 000234[0] 1736357943.645779 mapwrite 0x981a85 thru 0x985ed3 (0x444f bytes) 000236[1] 1736357943.674934 mapread 0x39cad5 thru 0x3a82f3 (0xb81f bytes) 000237[0] 1736357943.685613 mapwrite 0x8743b2 thru 0x87d507 (0x9156 bytes) 000238[0] 1736357943.697145 mapread 0x4fc206 thru 0x4fcee6 (0xce1 bytes) 000239[0] 1736357943.697426 write 0x1e71f2 thru 0x1eddc6 (0x6bd5 bytes) 000240[0] 1736357943.701358 trunc from 0x9c07d6 to 0x2bc88a (0x703f4b bytes) 000241[0] 1736357943.710638 write 0x461d18 thru 0x46df47 (0xc230 bytes) 000243[0] 1736357943.715405 mapwrite 0x687a0a thru 0x68cc80 (0x5277 bytes) 000244[1] 1736357943.737632 mapread 0x00e444 thru 0x017ba0 (0x975d bytes) 000246[0] 1736357943.746653 read 0x1831f7 thru 0x18aa85 (0x788f bytes) 000247[1] 1736357943.762827 mapread 0x210bef thru 0x21a31b (0x972d bytes) 000248[0] 1736357943.768051 mapread 0x2abd0f thru 0x2afa53 (0x3d45 bytes) 000249[0] 1736357943.770851 write 0x27e469 thru 0x28cdec (0xe984 bytes) 000250[0] 1736357943.779014 mapread 0x270423 thru 0x27ffb9 (0xfb97 bytes) 000252[0] 1736357943.795458 mapwrite 0x27b92d thru 0x27e427 (0x2afb bytes) 000254[1] 1736357943.807411 mapwrite 0x82b45f thru 0x8348f0 (0x9492 bytes) 000256[1] 1736357943.838976 mapread 0x21849e thru 0x220089 (0x7bec bytes) 000257[0] 1736357943.845576 read 0x7a5eb9 thru 0x7a6410 (0x558 bytes) 000258[1] 1736357943.853250 mapwrite 0x2ec2c1 thru 0x2eeee0 (0x2c20 bytes) 000259[0] 1736357943.866313 mapread 0x1b528c thru 0x1c2db0 (0xdb25 bytes) 000260[1] 1736357943.879039 mapwrite 0x229281 thru 0x23805d (0xeddd bytes) 000261[1] 1736357943.897803 mapread 0x3e198b thru 0x3e2467 (0xadd bytes) 000262[1] 1736357943.900281 write 0x37654c thru 0x384783 (0xe238 bytes) 000263[1] 1736357943.908456 trunc from 0x8348f1 to 0x68b598 (0x1a9358 bytes) 000264[1] 1736357943.915561 mapread 0x0bf6b4 thru 0x0cacfb (0xb648 bytes) 000265[1] 1736357943.924502 read 0x5cea4b thru 0x5dbadb (0xd091 bytes) 000266[1] 1736357943.927652 trunc from 0x68b599 to 0x0a186d (0x5e9d2b bytes) 000267[1] 1736357943.934932 trunc from 0x0a186e to 0x9d84ff (0x936c92 bytes) 000276[0] 1736357943.956382 trunc from 0x9d8500 to 0x981fb8 (0x56547 bytes) 000281[0] 1736357943.971920 write 0x920a6b thru 0x921f56 (0x14ec bytes) 000282[1] 1736357943.977621 mapwrite 0x8480bd thru 0x849560 (0x14a4 bytes) 000283[0] 1736357943.996972 trunc from 0x981fb9 to 0x3802e1 (0x601cd7 bytes) 000284[1] 1736357944.003244 write 0x7de1e3 thru 0x7e1c9c (0x3aba bytes) 000285[0] 1736357944.009460 mapwrite 0x2d22a2 thru 0x2d7fdf (0x5d3e bytes) 000286[0] 1736357944.029670 read 0x508eb0 thru 0x5151b3 (0xc304 bytes) 000287[0] 1736357944.046931 mapread 0x1cee5e thru 0x1d83d6 (0x9579 bytes) 000288[1] 1736357944.058609 mapread 0x666c67 thru 0x672ba3 (0xbf3d bytes) 000290[0] 1736357944.067454 read 0x7735f0 thru 0x77ff55 (0xc966 bytes) 000292[0] 1736357944.068745 mapwrite 0x8fadaa thru 0x9046da (0x9931 bytes) 000294[0] 1736357944.107041 read 0x4e14ee thru 0x4eceff (0xba12 bytes) 000297[0] 1736357944.128247 write 0x43d93d thru 0x4400c7 (0x278b bytes) 000300[1] 1736357944.133271 mapread 0x7e4d19 thru 0x7e8ab9 (0x3da1 bytes) 000301[0] 1736357944.142048 trunc from 0x9046db to 0x6006f1 (0x303fe9 bytes) 000302[1] 1736357944.151652 write 0x72d731 thru 0x7360d5 (0x89a5 bytes) 000304[1] 1736357944.161066 trunc from 0x7360d6 to 0x000927 (0x7357ae bytes) 000305[1] 1736357944.169121 mapwrite 0x5a2e40 thru 0x5add69 (0xaf2a bytes) 000308[0] 1736357944.203633 trunc from 0x5add6a to 0x4163b9 (0x1979b0 bytes) 000310[0] 1736357944.213636 read 0x3e7d05 thru 0x3f00d6 (0x83d2 bytes) 000312[1] 1736357944.219769 mapread 0x2a8b1d thru 0x2b6a91 (0xdf75 bytes) 000314[1] 1736357944.234139 mapread 0x10158c thru 0x10f46b (0xdee0 bytes) 000315[0] 1736357944.242408 read 0x126f55 thru 0x1317a1 (0xa84d bytes) 000316[0] 1736357944.248096 mapread 0x184405 thru 0x189670 (0x526c bytes) 000319[0] 1736357944.249498 trunc from 0x4163ba to 0x01dc4d (0x3f876c bytes) 000322[0] 1736357944.273509 mapread 0x00fcfc thru 0x0157a5 (0x5aaa bytes) 000323[1] 1736357944.284361 write 0x3b7fc6 thru 0x3c2c33 (0xac6e bytes) 000324[0] 1736357944.290871 read 0x07c046 thru 0x089759 (0xd714 bytes) 000326[0] 1736357944.312436 write 0x94bc78 thru 0x952407 (0x6790 bytes) 000330[0] 1736357944.323651 mapwrite 0x5c082a thru 0x5cbe23 (0xb5fa bytes) 000331[0] 1736357944.354817 trunc from 0x952408 to 0x7d3958 (0x17eaaf bytes) 000332[0] 1736357944.360313 mapread 0x43cacc thru 0x44696f (0x9ea4 bytes) 000333[0] 1736357944.360667 write 0x40dd14 thru 0x40f832 (0x1b1f bytes) 000338[1] 1736357944.378460 mapwrite 0x2a8a0c thru 0x2ac8a0 (0x3e95 bytes) 000339[1] 1736357944.402843 trunc from 0x7d3959 to 0x2e7c95 (0x4ebcc3 bytes) 000340[1] 1736357944.412406 write 0x0e5d63 thru 0x0e9682 (0x3920 bytes) 000341[1] 1736357944.418690 read 0x18a7a3 thru 0x195ecb (0xb729 bytes) 000343[1] 1736357944.430463 write 0x4cd815 thru 0x4d4feb (0x77d7 bytes) 000345[1] 1736357944.431635 read 0x131404 thru 0x136d07 (0x5904 bytes) 000346[1] 1736357944.438039 mapread 0x4d240f thru 0x4d48ac (0x249e bytes) 000348[0] 1736357944.438405 read 0x37c96f thru 0x3878e2 (0xaf74 bytes) 000349[1] 1736357944.454411 write 0x7bd133 thru 0x7caea5 (0xdd73 bytes) 000351[0] 1736357944.465814 read 0x5450f3 thru 0x54bb4e (0x6a5c bytes) 000355[1] 1736357944.482118 mapread 0x55ca78 thru 0x566cf0 (0xa279 bytes) 000356[0] 1736357944.494670 trunc from 0x7caea6 to 0x62f706 (0x19b79f bytes) 000357[1] 1736357944.512486 read 0x3d3716 thru 0x3df3c0 (0xbcab bytes) 000362[1] 1736357944.532731 trunc from 0x62f707 to 0x7bcb1e (0x18d418 bytes) 000365[1] 1736357944.545090 read 0x556da5 thru 0x55ccfa (0x5f56 bytes) 000366[1] 1736357944.555119 read 0x0cd64d thru 0x0d3415 (0x5dc9 bytes) 000368[1] 1736357944.577876 write 0x9ab541 thru 0x9b1ee1 (0x69a1 bytes) 000371[0] 1736357944.587096 write 0x3dd3a3 thru 0x3ecf15 (0xfb73 bytes) 000372[0] 1736357944.598645 trunc from 0x9b1ee2 to 0x32c520 (0x6859c1 bytes) 000375[0] 1736357944.610316 write 0x1e515b thru 0x1e9ec0 (0x4d66 bytes) 000376[1] 1736357944.612760 read 0x3204fa thru 0x320bd2 (0x6d9 bytes) 000378[0] 1736357944.619397 mapread 0x2166b6 thru 0x21ff70 (0x98bb bytes) 000379[1] 1736357944.628907 mapread 0x253534 thru 0x258b99 (0x5666 bytes) 000381[0] 1736357944.638594 write 0x873ff1 thru 0x87e4b5 (0xa4c5 bytes) 000382[1] 1736357944.642243 write 0x2db359 thru 0x2dbc6e (0x916 bytes) 000383[0] 1736357944.662710 mapwrite 0x32ecea thru 0x33435f (0x5676 bytes) 000385[1] 1736357944.717932 trunc from 0x87e4b6 to 0x768b99 (0x11591c bytes) 000388[1] 1736357944.731764 mapread 0x026a29 thru 0x02b72c (0x4d04 bytes) 000390[0] 1736357944.741369 mapwrite 0x2edab2 thru 0x2f29a8 (0x4ef7 bytes) 000391[0] 1736357944.776708 mapread 0x033ef7 thru 0x043d83 (0xfe8d bytes) 000392[1] 1736357944.800413 mapwrite 0x5da5f1 thru 0x5de72f (0x413f bytes) 000394[1] 1736357944.827201 write 0x19935f thru 0x199d1d (0x9bf bytes) 000396[1] 1736357944.838453 trunc from 0x768b9a to 0x6cc52a (0x9c66f bytes) 000397[1] 1736357944.847647 trunc from 0x6cc52b to 0x69d3a0 (0x2f18a bytes) 000398[1] 1736357944.852283 trunc from 0x69d3a1 to 0x044fbf (0x6583e1 bytes) 000399[1] 1736357944.859547 write 0x377a7c thru 0x37c074 (0x45f9 bytes) 000401[0] 1736357944.865493 write 0x56328f thru 0x563816 (0x588 bytes) 000402[1] 1736357944.873614 read 0x26ae6b thru 0x275978 (0xab0e bytes) 000403[0] 1736357944.894067 read 0x1cedb7 thru 0x1dc940 (0xdb8a bytes) 000404[0] 1736357944.899913 read 0x2b770c thru 0x2c2b7f (0xb474 bytes) 000405[1] 1736357944.910885 mapread 0x511971 thru 0x514f08 (0x3598 bytes) 000406[0] 1736357944.926973 write 0x188a79 thru 0x191039 (0x85c1 bytes) 000407[1] 1736357944.932130 mapread 0x2b9899 thru 0x2c5329 (0xba91 bytes) 000408[0] 1736357944.936403 mapread 0x29e7ad thru 0x2adecd (0xf721 bytes) 000409[0] 1736357944.956860 read 0x42dbc9 thru 0x43c1b4 (0xe5ec bytes) 000410[1] 1736357944.957248 mapwrite 0x0773b9 thru 0x07c404 (0x504c bytes) 000413[0] 1736357945.003230 trunc from 0x563817 to 0x74522f (0x1e1a19 bytes) 000414[0] 1736357945.009665 write 0x70c4f3 thru 0x710155 (0x3c63 bytes) 000415[1] 1736357945.011982 trunc from 0x745230 to 0x3c5b85 (0x37f6aa bytes) 000418[0] 1736357945.030701 trunc from 0x3c5b86 to 0x645f39 (0x2803b4 bytes) 000419[1] 1736357945.039386 read 0x4197fd thru 0x4291ac (0xf9b0 bytes) 000420[1] 1736357945.051413 trunc from 0x645f3a to 0x745f55 (0x10001c bytes) 000421[1] 1736357945.065753 trunc from 0x745f56 to 0x394075 (0x3b1ee0 bytes) 000422[0] 1736357945.073868 mapread 0x020daf thru 0x02a9b8 (0x9c0a bytes) 000424[1] 1736357945.095064 trunc from 0x394076 to 0x3d1077 (0x3d002 bytes) 000425[0] 1736357945.104856 mapread 0x01699d thru 0x01d962 (0x6fc6 bytes) 000427[0] 1736357945.118516 mapread 0x10750a thru 0x10ef43 (0x7a3a bytes) 000429[1] 1736357945.131116 mapread 0x0d28bd thru 0x0e0018 (0xd75c bytes) 000432[0] 1736357945.149494 read 0x311bde thru 0x315771 (0x3b94 bytes) 000434[0] 1736357945.155810 mapread 0x1bf6d1 thru 0x1c8fcc (0x98fc bytes) 000437[1] 1736357945.167295 write 0x264432 thru 0x270584 (0xc153 bytes) 000440[0] 1736357945.182711 mapwrite 0x0a4d35 thru 0x0b1d32 (0xcffe bytes) 000442[0] 1736357945.214885 write 0x381dba thru 0x38723b (0x5482 bytes) 000444[0] 1736357945.225604 mapread 0x0833ed thru 0x0843d3 (0xfe7 bytes) 000446[1] 1736357945.228265 mapread 0x251035 thru 0x25bbf2 (0xabbe bytes) 000449[0] 1736357945.253607 mapread 0x0b32da thru 0x0b81ae (0x4ed5 bytes) 000450[1] 1736357945.260265 mapread 0x06cf1e thru 0x0772b3 (0xa396 bytes) 000452[0] 1736357945.277607 mapwrite 0x8b4a27 thru 0x8b54da (0xab4 bytes) 000455[1] 1736357945.301296 mapwrite 0x6eeb4e thru 0x6f74d4 (0x8987 bytes) 000457[0] 1736357945.335869 read 0x4751d0 thru 0x47af4e (0x5d7f bytes) 000458[1] 1736357945.340451 mapread 0x18c836 thru 0x19754d (0xad18 bytes) 000459[0] 1736357945.375018 mapwrite 0x5738b6 thru 0x577b5f (0x42aa bytes) 000460[0] 1736357945.401915 trunc from 0x8b54db to 0x50e5d5 (0x3a6f05 bytes) 000461[1] 1736357945.414491 write 0x943898 thru 0x949802 (0x5f6b bytes) 000464[1] 1736357945.428844 write 0x2b88ee thru 0x2bb6f7 (0x2e0a bytes) 000465[0] 1736357945.435400 trunc from 0x949803 to 0x29e994 (0x6aae6e bytes) 000466[1] 1736357945.447579 write 0x86cb84 thru 0x87326e (0x66eb bytes) 000468[0] 1736357945.459172 read 0x5c94bf thru 0x5d087e (0x73c0 bytes) 000470[0] 1736357945.480111 trunc from 0x87326f to 0x6024c3 (0x270dab bytes) 000471[0] 1736357945.489462 trunc from 0x6024c4 to 0x56ea35 (0x93a8e bytes) 000473[0] 1736357945.493347 write 0x983725 thru 0x9921b7 (0xea93 bytes) 000474[0] 1736357945.499538 write 0x21f19c thru 0x225ac1 (0x6926 bytes) 000475[0] 1736357945.508282 mapread 0x2e8a81 thru 0x2e8cce (0x24e bytes) 000476[1] 1736357945.512787 trunc from 0x9921b8 to 0x5eb1de (0x3a6fd9 bytes) 000477[1] 1736357945.530776 mapwrite 0x4c93d0 thru 0x4d894e (0xf57f bytes) 000478[1] 1736357945.558829 mapread 0x44366b thru 0x4459ff (0x2395 bytes) 000482[1] 1736357945.569271 write 0x44c06e thru 0x45418b (0x811e bytes) 000484[1] 1736357945.579433 mapwrite 0x831b6c thru 0x83ed37 (0xd1cc bytes) 000486[0] 1736357945.624333 mapread 0x3aa08a thru 0x3acadb (0x2a52 bytes) 000489[1] 1736357945.638719 mapwrite 0x6bee96 thru 0x6c3604 (0x476f bytes) 000491[1] 1736357945.659491 trunc from 0x83ed38 to 0x512c3c (0x32c0fb bytes) 000493[1] 1736357945.671641 mapwrite 0x52dd1f thru 0x5386a5 (0xa987 bytes) 000497[1] 1736357945.712877 mapwrite 0x31967a thru 0x31b866 (0x21ed bytes) 000498[0] 1736357945.728320 write 0x8ae78d thru 0x8bdcda (0xf54e bytes) 000499[0] 1736357945.735358 read 0x388813 thru 0x38eb64 (0x6352 bytes) 000500[1] 1736357945.755908 read 0x2f72f2 thru 0x301240 (0x9f4f bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 7945 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read truncating to largest ever: 0x34736e 000003[0] 1736357945.977545 trunc from 00000000 to 0x34736d (0x34736e bytes) 000004[1] 1736357945.982305 mapwrite 0x1c7000 thru 0x1cea18 (0x7a19 bytes) 000009[1] 1736357946.016650 mapread 0x05e000 thru 0x060569 (0x256a bytes) 000012[1] 1736357946.025701 read 0x1fc000 thru 0x1fcab3 (0xab4 bytes) 000013[0] 1736357946.029322 mapwrite 0x0cf000 thru 0x0de8aa (0xf8ab bytes) 000014[1] 1736357946.055366 write 0x885000 thru 0x88b608 (0x6609 bytes) 000016[1] 1736357946.066054 trunc from 0x88b609 to 0x0e3cad (0x7a795b bytes) 000017[1] 1736357946.070873 trunc from 0x0e3cae to 0x1b5a92 (0xd1de5 bytes) truncating to largest ever: 0x3f453b 000018[1] 1736357946.076695 trunc from 0x1b5a93 to 0x3f453a (0x23eaa8 bytes) 000022[0] 1736357946.084208 read 0x118000 thru 0x11b38e (0x338f bytes) 000025[1] 1736357946.088264 mapwrite 0x770000 thru 0x77e5a7 (0xe5a8 bytes) 000027[1] 1736357946.113116 mapread 0x32c000 thru 0x32d8b5 (0x18b6 bytes) 000032[1] 1736357946.123181 write 0x069000 thru 0x06c050 (0x3051 bytes) 000035[1] 1736357946.132369 read 0x19d000 thru 0x1a7811 (0xa812 bytes) 000036[0] 1736357946.152254 mapread 0x3fe000 thru 0x403467 (0x5468 bytes) truncating to largest ever: 0x45a46b 000038[0] 1736357946.166286 trunc from 0x77e5a8 to 0x45a46a (0x32413d bytes) 000039[0] 1736357946.173624 mapread 0x32c000 thru 0x32c1e5 (0x1e6 bytes) 000040[1] 1736357946.176249 mapread 0x26e000 thru 0x26f7e3 (0x17e4 bytes) 000041[1] 1736357946.183203 read 0x3a3000 thru 0x3a951a (0x651b bytes) truncating to largest ever: 0x691855 000044[0] 1736357946.189937 trunc from 0x45a46b to 0x691854 (0x2373ea bytes) 000046[0] 1736357946.202228 mapwrite 0x502000 thru 0x50ebad (0xcbae bytes) 000047[1] 1736357946.224393 read 0x039000 thru 0x03f30f (0x6310 bytes) 000048[0] 1736357946.247982 read 0x37f000 thru 0x37f085 (0x86 bytes) 000049[0] 1736357946.258370 read 0x27f000 thru 0x282de0 (0x3de1 bytes) 000051[0] 1736357946.264017 write 0x2cc000 thru 0x2d92b4 (0xd2b5 bytes) 000052[1] 1736357946.273057 write 0x531000 thru 0x53f561 (0xe562 bytes) 000054[1] 1736357946.278733 read 0x5f7000 thru 0x5fc7f4 (0x57f5 bytes) 000056[0] 1736357946.285799 write 0x414000 thru 0x423af7 (0xfaf8 bytes) 000058[0] 1736357946.294954 mapwrite 0x7c9000 thru 0x7ce034 (0x5035 bytes) 000059[0] 1736357946.321984 read 0x097000 thru 0x09a54c (0x354d bytes) 000060[0] 1736357946.339246 mapwrite 0x19d000 thru 0x19d964 (0x965 bytes) 000063[1] 1736357946.360909 trunc from 0x7ce035 to 0x11d570 (0x6b0ac4 bytes) truncating to largest ever: 0x81f2f2 000064[1] 1736357946.378292 trunc from 0x11d571 to 0x81f2f1 (0x701d81 bytes) 000068[1] 1736357946.393220 write 0x69d000 thru 0x6a0d6e (0x3d6f bytes) 000070[0] 1736357946.398949 mapread 0x361000 thru 0x367f77 (0x6f78 bytes) 000072[1] 1736357946.413311 read 0x01b000 thru 0x022853 (0x7854 bytes) 000073[1] 1736357946.440674 mapwrite 0x2aa000 thru 0x2acce2 (0x2ce3 bytes) 000075[0] 1736357946.489190 mapread 0x5f5000 thru 0x5fe532 (0x9533 bytes) 000076[1] 1736357946.505656 mapread 0x806000 thru 0x80bfcd (0x5fce bytes) 000077[1] 1736357946.513370 trunc from 0x81f2f2 to 0x0b13f1 (0x76df00 bytes) 000079[0] 1736357946.520477 mapwrite 0x740000 thru 0x747015 (0x7016 bytes) 000081[1] 1736357946.550717 read 0x31e000 thru 0x322ee5 (0x4ee6 bytes) 000082[0] 1736357946.570746 mapwrite 0x056000 thru 0x05af6a (0x4f6b bytes) 000083[1] 1736357946.598046 trunc from 0x747016 to 0x406291 (0x340d84 bytes) 000084[1] 1736357946.603715 mapwrite 0x600000 thru 0x609944 (0x9945 bytes) 000087[1] 1736357946.628842 read 0x5c5000 thru 0x5cd211 (0x8212 bytes) 000088[1] 1736357946.631910 write 0x327000 thru 0x32ad65 (0x3d66 bytes) 000091[1] 1736357946.633761 trunc from 0x609945 to 0x47ea05 (0x18af3f bytes) 000092[1] 1736357946.638069 trunc from 0x47ea06 to 0x3252ba (0x15974b bytes) 000093[1] 1736357946.641387 trunc from 0x3252bb to 0x33429f (0xefe5 bytes) 000094[0] 1736357946.645457 trunc from 0x3342a0 to 0x5a993e (0x27569f bytes) 000095[0] 1736357946.655752 mapwrite 0x6e1000 thru 0x6efcf5 (0xecf6 bytes) 000097[1] 1736357946.696683 mapread 0x5ee000 thru 0x5fa5a6 (0xc5a7 bytes) 000099[0] 1736357946.718714 mapwrite 0x5c0000 thru 0x5cd774 (0xd775 bytes) 000101[0] 1736357946.759039 mapwrite 0x78d000 thru 0x7905e5 (0x35e6 bytes) 000102[1] 1736357946.784753 write 0x826000 thru 0x829dd6 (0x3dd7 bytes) 000107[0] 1736357946.796073 write 0x1f9000 thru 0x204613 (0xb614 bytes) 000112[1] 1736357946.815502 read 0x1db000 thru 0x1e0d72 (0x5d73 bytes) 000113[0] 1736357946.839920 trunc from 0x829dd7 to 0x1139a6 (0x716430 bytes) 000115[0] 1736357946.853376 mapread 0x03a000 thru 0x049e41 (0xfe42 bytes) 000116[0] 1736357946.871187 mapread 0x010000 thru 0x01d445 (0xd446 bytes) 000118[1] 1736357946.885072 write 0x0a0000 thru 0x0a807a (0x807b bytes) 000123[1] 1736357946.898056 mapread 0x0b8000 thru 0x0c38cd (0xb8ce bytes) 000125[0] 1736357946.913914 mapwrite 0x449000 thru 0x458380 (0xf381 bytes) 000127[0] 1736357946.969639 trunc from 0x458381 to 0x7afc55 (0x3578d5 bytes) 000128[0] 1736357946.980195 mapread 0x484000 thru 0x486866 (0x2867 bytes) 000129[1] 1736357946.987044 write 0x46c000 thru 0x47b467 (0xf468 bytes) 000130[0] 1736357946.997656 write 0x695000 thru 0x69dada (0x8adb bytes) 000134[0] 1736357947.013529 mapread 0x0c3000 thru 0x0c39a1 (0x9a2 bytes) 000136[1] 1736357947.016027 mapread 0x69b000 thru 0x6a0466 (0x5467 bytes) 000137[1] 1736357947.034428 write 0x0ad000 thru 0x0ba76e (0xd76f bytes) 000139[1] 1736357947.047694 write 0x33d000 thru 0x33e926 (0x1927 bytes) 000140[0] 1736357947.052012 mapread 0x52d000 thru 0x53091d (0x391e bytes) 000141[0] 1736357947.065282 trunc from 0x7afc56 to 0x77c65c (0x335f9 bytes) 000143[0] 1736357947.071405 trunc from 0x77c65d to 0x537a27 (0x244c35 bytes) 000145[1] 1736357947.079604 mapread 0x45e000 thru 0x4608ae (0x28af bytes) 000146[0] 1736357947.087090 mapread 0x33a000 thru 0x342f52 (0x8f53 bytes) 000147[1] 1736357947.098999 mapwrite 0x177000 thru 0x17abf5 (0x3bf6 bytes) 000148[1] 1736357947.120302 read 0x18c000 thru 0x199ec7 (0xdec8 bytes) 000151[0] 1736357947.143041 trunc from 0x537a28 to 0x02b26d (0x50c7ba bytes) 000152[0] 1736357947.155086 mapread 0x013000 thru 0x01fdb9 (0xcdba bytes) 000157[0] 1736357947.181837 trunc from 0x02b26e to 0x2c1e9b (0x296c2e bytes) 000158[0] 1736357947.189209 mapread 0x230000 thru 0x23bd5d (0xbd5e bytes) 000159[1] 1736357947.214370 mapread 0x262000 thru 0x268a56 (0x6a57 bytes) 000160[1] 1736357947.232335 mapwrite 0x958000 thru 0x95dd6f (0x5d70 bytes) 000161[0] 1736357947.278770 read 0x05f000 thru 0x06aeeb (0xbeec bytes) truncating to largest ever: 0x85753b 000162[0] 1736357947.316408 trunc from 0x95dd70 to 0x85753a (0x106835 bytes) 000163[0] 1736357947.337133 mapread 0x2b8000 thru 0x2c597e (0xd97f bytes) 000165[0] 1736357947.357654 mapread 0x240000 thru 0x2435e9 (0x35ea bytes) 000166[0] 1736357947.365740 mapwrite 0x074000 thru 0x080b42 (0xcb43 bytes) 000167[1] 1736357947.394943 write 0x95b000 thru 0x95d5e4 (0x25e5 bytes) 000170[0] 1736357947.400778 mapwrite 0x02d000 thru 0x039ba3 (0xcba4 bytes) 000172[1] 1736357947.432003 write 0x025000 thru 0x032bf9 (0xdbfa bytes) 000173[0] 1736357947.438747 mapwrite 0x175000 thru 0x1822e6 (0xd2e7 bytes) 000175[1] 1736357947.485863 mapwrite 0x672000 thru 0x67cc45 (0xac46 bytes) 000176[0] 1736357947.517180 mapread 0x3d4000 thru 0x3dd3c8 (0x93c9 bytes) 000177[0] 1736357947.532086 write 0x8b1000 thru 0x8b7ccc (0x6ccd bytes) truncating to largest ever: 0x9ca4c0 000178[0] 1736357947.536545 trunc from 0x95d5e5 to 0x9ca4bf (0x6cedb bytes) 000179[0] 1736357947.541868 read 0x19b000 thru 0x1a1408 (0x6409 bytes) 000184[1] 1736357947.576303 read 0x6d2000 thru 0x6def3a (0xcf3b bytes) 000186[1] 1736357947.602028 mapread 0x675000 thru 0x678110 (0x3111 bytes) 000187[1] 1736357947.608533 trunc from 0x9ca4c0 to 0x2f6f5d (0x6d3562 bytes) 000188[0] 1736357947.622118 mapread 0x0cb000 thru 0x0cdb5e (0x2b5f bytes) 000191[1] 1736357947.631340 trunc from 0x2f6f5e to 0x183a4d (0x173510 bytes) 000192[0] 1736357947.637051 trunc from 0x183a4e to 0x2338ea (0xafe9d bytes) 000195[0] 1736357947.647598 mapwrite 0x20a000 thru 0x20d8d1 (0x38d2 bytes) 000197[1] 1736357947.662975 trunc from 0x2338eb to 0x2eb607 (0xb7d1d bytes) 000200[1] 1736357947.671682 read 0x072000 thru 0x08134a (0xf34b bytes) 000201[1] 1736357947.680746 write 0x8ad000 thru 0x8b11aa (0x41ab bytes) 000203[0] 1736357947.682202 mapread 0x4a6000 thru 0x4ae27f (0x8280 bytes) 000204[1] 1736357947.705225 write 0x70b000 thru 0x711b1f (0x6b20 bytes) 000207[0] 1736357947.733340 write 0x9f4000 thru 0x9ff51f (0xb520 bytes) 000209[1] 1736357947.747376 write 0x93d000 thru 0x943bc7 (0x6bc8 bytes) 000214[0] 1736357947.774032 mapread 0x95d000 thru 0x95e7dc (0x17dd bytes) 000215[0] 1736357947.784718 trunc from 0x9ff520 to 0x7d9caf (0x225870 bytes) 000216[0] 1736357947.793441 mapread 0x3e2000 thru 0x3e5b8b (0x3b8c bytes) 000220[0] 1736357947.811897 write 0x38e000 thru 0x39ce0b (0xee0c bytes) 000223[0] 1736357947.815526 write 0x6bb000 thru 0x6bbd6e (0xd6f bytes) 000224[0] 1736357947.819863 mapwrite 0x842000 thru 0x843b08 (0x1b09 bytes) 000225[0] 1736357947.841904 mapwrite 0x59e000 thru 0x5a1adb (0x3adc bytes) truncating to largest ever: 0x9da1e8 000226[1] 1736357947.857785 trunc from 0x843b09 to 0x9da1e7 (0x1966df bytes) 000227[1] 1736357947.866705 write 0x9fe000 thru 0x9ffd02 (0x1d03 bytes) 000232[1] 1736357947.876057 write 0x3f4000 thru 0x3fdce5 (0x9ce6 bytes) 000233[1] 1736357947.884139 trunc from 0x9ffd03 to 0x8af641 (0x1506c1 bytes) 000234[0] 1736357947.891135 trunc from 0x8af642 to 0x4564af (0x459192 bytes) 000236[1] 1736357947.900905 mapread 0x136000 thru 0x13b41c (0x541d bytes) 000237[0] 1736357947.909423 read 0x386000 thru 0x3909fd (0xa9fe bytes) 000238[1] 1736357947.915607 mapread 0x3d7000 thru 0x3e2e7f (0xbe80 bytes) 000239[1] 1736357947.927407 trunc from 0x4564b0 to 0x4dde55 (0x879a6 bytes) 000240[0] 1736357947.938568 read 0x232000 thru 0x240ae5 (0xeae6 bytes) 000241[1] 1736357947.953156 write 0x319000 thru 0x31f54e (0x654f bytes) 000246[0] 1736357947.962024 mapwrite 0x88b000 thru 0x897e0d (0xce0e bytes) 000247[0] 1736357947.999811 mapread 0x27f000 thru 0x281b67 (0x2b68 bytes) 000253[1] 1736357948.011838 mapread 0x5e2000 thru 0x5e71e8 (0x51e9 bytes) 000254[0] 1736357948.024986 read 0x652000 thru 0x65832e (0x632f bytes) 000255[1] 1736357948.032951 read 0x729000 thru 0x735960 (0xc961 bytes) 000256[0] 1736357948.040447 mapwrite 0x966000 thru 0x968161 (0x2162 bytes) 000257[0] 1736357948.067675 read 0x703000 thru 0x705075 (0x2076 bytes) 000258[0] 1736357948.079420 read 0x7f2000 thru 0x7f9a22 (0x7a23 bytes) 000259[0] 1736357948.080570 mapwrite 0x462000 thru 0x46f600 (0xd601 bytes) 000261[0] 1736357948.106013 trunc from 0x968162 to 0x2600c1 (0x7080a0 bytes) 000262[0] 1736357948.115616 write 0x446000 thru 0x4534e5 (0xd4e6 bytes) 000263[1] 1736357948.117705 write 0x133000 thru 0x13aa3a (0x7a3b bytes) 000265[1] 1736357948.130045 trunc from 0x4534e6 to 0x85427c (0x400d97 bytes) 000267[1] 1736357948.141272 read 0x1bc000 thru 0x1cba97 (0xfa98 bytes) 000270[0] 1736357948.166008 mapread 0x66f000 thru 0x671ab4 (0x2ab5 bytes) 000274[1] 1736357948.180759 mapread 0x808000 thru 0x817780 (0xf781 bytes) 000275[1] 1736357948.197183 mapread 0x1ef000 thru 0x1fefc0 (0xffc1 bytes) 000278[0] 1736357948.214770 mapread 0x7bb000 thru 0x7c8159 (0xd15a bytes) 000280[1] 1736357948.230264 write 0x28f000 thru 0x294874 (0x5875 bytes) 000284[1] 1736357948.241741 mapread 0x2d9000 thru 0x2dd3bc (0x43bd bytes) 000285[0] 1736357948.247027 mapwrite 0x8ba000 thru 0x8c459c (0xa59d bytes) 000288[1] 1736357948.284386 write 0x8d4000 thru 0x8d745b (0x345c bytes) 000289[1] 1736357948.290301 mapwrite 0x7aa000 thru 0x7b2ab2 (0x8ab3 bytes) 000293[1] 1736357948.322495 mapwrite 0x18d000 thru 0x199038 (0xc039 bytes) 000294[0] 1736357948.350702 write 0x3aa000 thru 0x3adb72 (0x3b73 bytes) 000295[1] 1736357948.356815 mapwrite 0x135000 thru 0x144309 (0xf30a bytes) 000297[1] 1736357948.389158 trunc from 0x8d745c to 0x6a50c4 (0x232397 bytes) 000298[0] 1736357948.395830 read 0x577000 thru 0x57d39b (0x639c bytes) 000302[1] 1736357948.405030 trunc from 0x6a50c5 to 0x306d2f (0x39e395 bytes) 000304[0] 1736357948.411399 mapwrite 0x09f000 thru 0x0aabb2 (0xbbb3 bytes) 000305[0] 1736357948.438352 trunc from 0x306d30 to 0x1577d5 (0x1af55a bytes) 000306[1] 1736357948.442649 mapwrite 0x4cc000 thru 0x4d2fe8 (0x6fe9 bytes) 000308[0] 1736357948.465027 write 0x3c3000 thru 0x3c54b2 (0x24b3 bytes) 000309[1] 1736357948.470721 mapread 0x3ce000 thru 0x3d9ff9 (0xbffa bytes) 000311[0] 1736357948.486309 write 0x8de000 thru 0x8e7744 (0x9745 bytes) 000313[0] 1736357948.493982 mapwrite 0x9dd000 thru 0x9de53d (0x153e bytes) 000314[0] 1736357948.518100 mapwrite 0x851000 thru 0x8521d9 (0x11da bytes) 000317[0] 1736357948.528091 read 0x19f000 thru 0x1a61d3 (0x71d4 bytes) 000318[0] 1736357948.542342 trunc from 0x9de53e to 0x9a981d (0x34d20 bytes) 000319[1] 1736357948.545730 trunc from 0x9a981e to 0x4c9781 (0x4e009c bytes) 000320[0] 1736357948.561788 mapwrite 0x696000 thru 0x6a0120 (0xa121 bytes) 000321[1] 1736357948.589041 write 0x347000 thru 0x354159 (0xd15a bytes) 000323[1] 1736357948.594184 mapwrite 0x867000 thru 0x86da3a (0x6a3b bytes) 000326[0] 1736357948.624368 read 0x636000 thru 0x63e4e2 (0x84e3 bytes) 000328[1] 1736357948.638416 mapwrite 0x7ec000 thru 0x7f3a85 (0x7a86 bytes) 000330[1] 1736357948.666865 write 0x1ad000 thru 0x1b61c5 (0x91c6 bytes) 000331[0] 1736357948.668330 mapwrite 0x9f4000 thru 0x9fcf91 (0x8f92 bytes) 000333[0] 1736357948.693924 trunc from 0x9fcf92 to 0x86961f (0x193972 bytes) 000334[1] 1736357948.700308 trunc from 0x869620 to 0x827f81 (0x4169e bytes) 000336[1] 1736357948.709744 read 0x75e000 thru 0x75fa99 (0x1a9a bytes) 000337[0] 1736357948.714701 read 0x318000 thru 0x31f39b (0x739c bytes) 000338[1] 1736357948.728671 write 0x137000 thru 0x13c451 (0x5452 bytes) 000340[1] 1736357948.738296 mapwrite 0x434000 thru 0x43a697 (0x6698 bytes) 000341[1] 1736357948.759210 mapread 0x54e000 thru 0x55423d (0x623e bytes) 000342[1] 1736357948.768677 mapwrite 0x65d000 thru 0x66a7b1 (0xd7b2 bytes) 000345[1] 1736357948.797146 mapread 0x6cb000 thru 0x6d443e (0x943f bytes) 000346[1] 1736357948.807210 trunc from 0x827f82 to 0x6a4c3f (0x183342 bytes) 000349[1] 1736357948.818139 read 0x46c000 thru 0x47764c (0xb64d bytes) 000350[1] 1736357948.826942 read 0x4d0000 thru 0x4da45e (0xa45f bytes) 000351[0] 1736357948.827238 write 0x495000 thru 0x496559 (0x155a bytes) 000352[0] 1736357948.839651 mapread 0x51b000 thru 0x520a73 (0x5a74 bytes) 000353[1] 1736357948.846662 read 0x0df000 thru 0x0e719e (0x819f bytes) 000359[1] 1736357948.871892 mapread 0x5e4000 thru 0x5ee2b9 (0xa2ba bytes) 000362[0] 1736357948.872466 read 0x3e2000 thru 0x3e7dd3 (0x5dd4 bytes) 000365[0] 1736357948.882046 write 0x628000 thru 0x62e06a (0x606b bytes) 000366[0] 1736357948.892406 trunc from 0x6a4c40 to 0x2c305f (0x3e1be0 bytes) 000367[1] 1736357948.899354 write 0x4e0000 thru 0x4e2869 (0x286a bytes) 000369[0] 1736357948.905084 read 0x0fa000 thru 0x0fa25f (0x260 bytes) 000371[1] 1736357948.921379 write 0x08f000 thru 0x097c6c (0x8c6d bytes) 000373[0] 1736357948.928392 mapwrite 0x51d000 thru 0x521729 (0x472a bytes) 000377[0] 1736357948.957146 mapwrite 0x588000 thru 0x591fe1 (0x9fe2 bytes) 000380[1] 1736357948.992756 read 0x2c8000 thru 0x2ccab0 (0x4ab1 bytes) 000381[0] 1736357949.004220 read 0x2c7000 thru 0x2d3942 (0xc943 bytes) 000383[0] 1736357949.015238 read 0x017000 thru 0x021b91 (0xab92 bytes) 000385[0] 1736357949.024907 mapread 0x46c000 thru 0x46e5b6 (0x25b7 bytes) 000387[0] 1736357949.025364 mapwrite 0x8ec000 thru 0x8f22d3 (0x62d4 bytes) 000388[1] 1736357949.062418 mapwrite 0x2af000 thru 0x2ba896 (0xb897 bytes) 000389[1] 1736357949.104460 write 0x9b4000 thru 0x9be31d (0xa31e bytes) 000392[0] 1736357949.108820 mapwrite 0x0fa000 thru 0x0ff210 (0x5211 bytes) 000395[0] 1736357949.133132 write 0x7df000 thru 0x7ebd40 (0xcd41 bytes) 000399[1] 1736357949.141245 write 0x0f9000 thru 0x0ffc07 (0x6c08 bytes) 000401[0] 1736357949.158272 write 0x728000 thru 0x7294e5 (0x14e6 bytes) 000402[1] 1736357949.164526 mapread 0x8f4000 thru 0x8f42cd (0x2ce bytes) 000404[1] 1736357949.171300 trunc from 0x9be31e to 0x1190b4 (0x8a5269 bytes) 000407[0] 1736357949.185631 mapread 0x0e0000 thru 0x0ef185 (0xf186 bytes) 000411[1] 1736357949.208278 mapwrite 0x38c000 thru 0x39546b (0x946c bytes) 000414[1] 1736357949.241157 write 0x7fe000 thru 0x8083e6 (0xa3e7 bytes) 000417[1] 1736357949.249114 trunc from 0x8083e7 to 0x15336f (0x6b5077 bytes) 000418[0] 1736357949.253033 mapread 0x11a000 thru 0x11eaa5 (0x4aa6 bytes) 000421[0] 1736357949.262256 read 0x0f6000 thru 0x104cc3 (0xecc4 bytes) 000425[0] 1736357949.265797 write 0x5a8000 thru 0x5a9732 (0x1733 bytes) 000427[1] 1736357949.271680 write 0x552000 thru 0x553090 (0x1091 bytes) 000430[1] 1736357949.281353 read 0x27b000 thru 0x28850b (0xd50c bytes) 000431[0] 1736357949.293559 write 0x1fb000 thru 0x2050ee (0xa0ef bytes) 000432[0] 1736357949.308003 read 0x252000 thru 0x25585d (0x385e bytes) 000434[0] 1736357949.322621 mapread 0x1f8000 thru 0x200670 (0x8671 bytes) 000436[0] 1736357949.325887 mapwrite 0x3fd000 thru 0x40bbf9 (0xebfa bytes) 000437[1] 1736357949.341917 read 0x0f4000 thru 0x0f67d8 (0x27d9 bytes) 000439[0] 1736357949.362474 mapread 0x291000 thru 0x29e3af (0xd3b0 bytes) 000440[0] 1736357949.376403 read 0x523000 thru 0x531839 (0xe83a bytes) 000441[1] 1736357949.380824 mapwrite 0x5f8000 thru 0x5ffbb5 (0x7bb6 bytes) 000442[0] 1736357949.410619 read 0x089000 thru 0x08d555 (0x4556 bytes) 000443[1] 1736357949.432511 read 0x400000 thru 0x401260 (0x1261 bytes) 000444[1] 1736357949.440859 trunc from 0x5ffbb6 to 0x96e86c (0x36ecb7 bytes) 000445[0] 1736357949.456285 write 0x73c000 thru 0x742012 (0x6013 bytes) 000446[0] 1736357949.465087 mapwrite 0x8c0000 thru 0x8c7930 (0x7931 bytes) 000449[1] 1736357949.495985 mapread 0x3c7000 thru 0x3cd88a (0x688b bytes) 000451[0] 1736357949.521110 trunc from 0x96e86d to 0x86c8e7 (0x101f85 bytes) 000453[0] 1736357949.536767 write 0x229000 thru 0x22f976 (0x6977 bytes) 000454[1] 1736357949.539367 mapwrite 0x6d1000 thru 0x6d9409 (0x840a bytes) 000455[1] 1736357949.566500 read 0x768000 thru 0x77482b (0xc82c bytes) 000459[0] 1736357949.578140 trunc from 0x86c8e8 to 0x13eff1 (0x72d8f6 bytes) 000461[0] 1736357949.589797 write 0x209000 thru 0x210fac (0x7fad bytes) 000463[0] 1736357949.590897 mapread 0x096000 thru 0x0976f4 (0x16f5 bytes) 000465[0] 1736357949.593425 read 0x07a000 thru 0x0848d6 (0xa8d7 bytes) 000467[0] 1736357949.602872 mapread 0x1d2000 thru 0x1d2321 (0x322 bytes) 000468[0] 1736357949.605889 read 0x1bc000 thru 0x1c5384 (0x9385 bytes) 000470[0] 1736357949.609513 read 00000000 thru 0x00a77b (0xa77c bytes) 000472[0] 1736357949.616600 mapwrite 0x7bf000 thru 0x7c3ba5 (0x4ba6 bytes) 000473[0] 1736357949.653416 mapwrite 0x9a5000 thru 0x9a8ce4 (0x3ce5 bytes) 000476[0] 1736357949.686282 mapwrite 0x73e000 thru 0x742393 (0x4394 bytes) 000479[0] 1736357949.709643 mapread 0x3bc000 thru 0x3c54b3 (0x94b4 bytes) 000480[0] 1736357949.711165 mapread 0x0fd000 thru 0x10661c (0x961d bytes) 000481[0] 1736357949.711380 mapread 0x649000 thru 0x6581ec (0xf1ed bytes) 000482[0] 1736357949.714452 read 0x3c8000 thru 0x3d3bf7 (0xbbf8 bytes) 000483[0] 1736357949.714658 trunc from 0x9a8ce5 to 0x31b80e (0x68d4d6 bytes) 000484[1] 1736357949.729262 mapread 0x147000 thru 0x156c55 (0xfc56 bytes) 000485[1] 1736357949.750580 mapread 0x250000 thru 0x2576a2 (0x76a3 bytes) 000486[0] 1736357949.757739 read 0x179000 thru 0x179914 (0x915 bytes) 000487[1] 1736357949.764619 write 0x3ca000 thru 0x3ccfca (0x2fcb bytes) 000490[1] 1736357949.774542 read 0x092000 thru 0x0a172e (0xf72f bytes) 000492[1] 1736357949.790049 mapwrite 0x26c000 thru 0x271891 (0x5892 bytes) 000494[1] 1736357949.803584 write 0x3a4000 thru 0x3b1f8d (0xdf8e bytes) 000496[0] 1736357949.807776 mapwrite 0x0a2000 thru 0x0acd3d (0xad3e bytes) 000497[0] 1736357949.848984 trunc from 0x3ccfcb to 0x823e78 (0x456eae bytes) 000499[0] 1736357949.859575 trunc from 0x823e79 to 0x345673 (0x4de805 bytes) All operations completed A-OK! PASS 16a (10s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 12:39:10 (1736357950) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0667855 s, 31.4 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0567268 s, 37.0 MB/s Chance of close/open is 1 in 50 Seed set to 7951 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x80c091 000001[1] 1736357951.581827 trunc from 00000000 to 0x80c090 (0x80c091 bytes) 000004[1] 1736357951.594513 write 0x07fc2a thru 0x081509 (0x18e0 bytes) 000006[1] 1736357951.600163 read 0x70a14c thru 0x70b921 (0x17d6 bytes) 000007[0] 1736357951.604673 trunc from 0x80c091 to 0x162b8e (0x6a9502 bytes) 000011[0] 1736357951.620415 read 0x06d3d7 thru 0x06e757 (0x1381 bytes) 000012[0] 1736357951.624903 write 0x4daaec thru 0x4dbbb5 (0x10ca bytes) 000016[0] 1736357951.632473 write 0x3e6f8f thru 0x3e70d9 (0x14b bytes) 000017[1] 1736357951.634502 trunc from 0x4dbbb6 to 0x47617a (0x65a3b bytes) 000018[1] 1736357951.646480 trunc from 0x47617b to 0x5affd8 (0x139e5e bytes) 000019[1] 1736357951.650835 mapread 0x183ea8 thru 0x1848f3 (0xa4c bytes) 000020[1] 1736357951.656840 mapread 0x19408a thru 0x1959e1 (0x1958 bytes) 000021[1] 1736357951.662129 mapwrite 0x76bb31 thru 0x76c8b0 (0xd80 bytes) 000022[1] 1736357951.691353 write 0x7a8ea7 thru 0x7a96ea (0x844 bytes) 000023[0] 1736357951.691795 mapwrite 0x246383 thru 0x24670c (0x38a bytes) 000024[1] 1736357951.715686 write 0x1476a7 thru 0x14823e (0xb98 bytes) 000026[1] 1736357951.724174 mapwrite 0x951e7a thru 0x953bfe (0x1d85 bytes) 000027[0] 1736357951.755640 write 0x464794 thru 0x4659df (0x124c bytes) 000028[0] 1736357951.764881 write 0x371f61 thru 0x37212e (0x1ce bytes) 000029[0] 1736357951.774399 write 0x25d772 thru 0x25dec0 (0x74f bytes) 000032[0] 1736357951.786408 write 0x8877b7 thru 0x888b48 (0x1392 bytes) 000034[0] 1736357951.794018 write 0x811a4b thru 0x812eea (0x14a0 bytes) 000036[0] 1736357951.801589 read 0x5d7f23 thru 0x5d89cc (0xaaa bytes) 000038[0] 1736357951.828138 mapread 0x59af54 thru 0x59ba74 (0xb21 bytes) 000039[0] 1736357951.835069 mapwrite 0x2dfb4e thru 0x2e12dd (0x1790 bytes) 000040[0] 1736357951.862164 mapread 0x63c6b6 thru 0x63cf4b (0x896 bytes) 000041[0] 1736357951.866384 read 0x940e9c thru 0x941914 (0xa79 bytes) 000043[1] 1736357951.873491 mapread 0x1160ed thru 0x1177ae (0x16c2 bytes) 000045[1] 1736357951.887032 write 0x3ef1f6 thru 0x3efd6b (0xb76 bytes) 000046[1] 1736357951.892831 write 0x8fd2a5 thru 0x8fdf80 (0xcdc bytes) 000047[0] 1736357951.900415 trunc from 0x953bff to 0x3a5bad (0x5ae051 bytes) 000048[1] 1736357951.910484 mapread 0x11f9af thru 0x120129 (0x77b bytes) 000049[1] 1736357951.917945 read 0x37e14f thru 0x37f812 (0x16c4 bytes) 000051[0] 1736357951.926093 write 0x00e0e0 thru 0x00e9d3 (0x8f4 bytes) 000052[0] 1736357951.927767 mapwrite 0x4d1984 thru 0x4d3935 (0x1fb2 bytes) 000053[0] 1736357951.957439 mapread 0x4252a6 thru 0x42615e (0xeb9 bytes) 000055[0] 1736357951.966150 trunc from 0x4d3936 to 0x05aea8 (0x478a8d bytes) 000056[0] 1736357951.975564 trunc from 0x05aea9 to 0x7fbe84 (0x7a0fdc bytes) 000057[1] 1736357951.981627 mapwrite 0x385d6c thru 0x387837 (0x1acc bytes) 000058[0] 1736357952.014444 mapwrite 0x49b506 thru 0x49b747 (0x242 bytes) truncating to largest ever: 0x9de786 000059[1] 1736357952.032172 trunc from 0x7fbe85 to 0x9de785 (0x1e2901 bytes) 000060[0] 1736357952.039640 mapwrite 0x4cf67c thru 0x4cf8a4 (0x229 bytes) 000062[0] 1736357952.063987 mapread 0x3d4269 thru 0x3d4bda (0x972 bytes) 000064[0] 1736357952.081005 mapwrite 0x4f8879 thru 0x4fa01f (0x17a7 bytes) 000065[0] 1736357952.097191 mapread 0x7dd855 thru 0x7df743 (0x1eef bytes) 000067[0] 1736357952.101243 mapwrite 0x5c9dbb thru 0x5ca9f1 (0xc37 bytes) 000068[1] 1736357952.117179 trunc from 0x9de786 to 0x5462d4 (0x4984b1 bytes) 000070[1] 1736357952.130886 mapread 0x279de1 thru 0x27b124 (0x1344 bytes) 000071[1] 1736357952.145756 trunc from 0x5462d5 to 0x347b99 (0x1fe73b bytes) 000073[0] 1736357952.157565 trunc from 0x347b9a to 0x3a39cb (0x5be32 bytes) 000074[1] 1736357952.165221 read 0x1d3d57 thru 0x1d46ab (0x955 bytes) 000078[1] 1736357952.184931 read 0x1513f0 thru 0x151886 (0x497 bytes) 000079[0] 1736357952.195178 write 0x81c4cc thru 0x81d234 (0xd69 bytes) 000080[1] 1736357952.198749 write 0x4d6429 thru 0x4d7f1a (0x1af2 bytes) 000087[1] 1736357952.235907 write 0x35f7a7 thru 0x360882 (0x10dc bytes) 000088[0] 1736357952.246553 write 0x125ec2 thru 0x127008 (0x1147 bytes) 000092[0] 1736357952.267778 write 0x8f9436 thru 0x8f94de (0xa9 bytes) 000093[0] 1736357952.280233 trunc from 0x8f94df to 0x0a6535 (0x852fa9 bytes) 000094[0] 1736357952.286549 read 0x018fa8 thru 0x01a804 (0x185d bytes) 000096[0] 1736357952.297369 trunc from 0x0a6536 to 0x63c966 (0x596431 bytes) 000097[1] 1736357952.308603 write 0x05516d thru 0x055bca (0xa5e bytes) 000103[1] 1736357952.336555 mapwrite 0x87d835 thru 0x87dcd2 (0x49e bytes) 000104[0] 1736357952.362686 read 0x0560f4 thru 0x056892 (0x79f bytes) 000106[0] 1736357952.372043 read 0x59e71f thru 0x59fb54 (0x1436 bytes) 000108[0] 1736357952.389102 read 0x4cd2fa thru 0x4cdccf (0x9d6 bytes) 000110[0] 1736357952.394408 trunc from 0x87dcd3 to 0x0903f5 (0x7ed8dd bytes) 000113[1] 1736357952.414350 read 0x08d8d8 thru 0x08e2af (0x9d8 bytes) 000116[1] 1736357952.425403 read 0x08aeb8 thru 0x08bda8 (0xef1 bytes) 000117[0] 1736357952.430762 write 0x73e64a thru 0x73f32c (0xce3 bytes) 000119[0] 1736357952.437014 trunc from 0x73f32d to 0x647266 (0xf80c6 bytes) 000120[0] 1736357952.447690 trunc from 0x647267 to 0x1ef9c2 (0x4578a4 bytes) 000121[1] 1736357952.456322 read 0x102cba thru 0x103985 (0xccc bytes) 000122[1] 1736357952.469002 trunc from 0x1ef9c3 to 0x0a9e02 (0x145bc0 bytes) 000123[1] 1736357952.479484 mapread 0x0403fb thru 0x040445 (0x4b bytes) 000126[1] 1736357952.487018 mapwrite 0x37103a thru 0x3728a2 (0x1869 bytes) 000128[0] 1736357952.517705 mapwrite 0x543d32 thru 0x544f34 (0x1203 bytes) 000129[1] 1736357952.541110 mapwrite 0x2170b4 thru 0x217656 (0x5a3 bytes) 000130[0] 1736357952.563091 trunc from 0x544f35 to 0x69b771 (0x15683d bytes) 000138[0] 1736357952.578314 read 0x1518fa thru 0x153833 (0x1f3a bytes) 000141[0] 1736357952.595269 read 0x4396e7 thru 0x43b419 (0x1d33 bytes) 000143[0] 1736357952.601812 read 0x243dfc thru 0x243e26 (0x2b bytes) 000144[0] 1736357952.602057 write 0x4a1d85 thru 0x4a296d (0xbe9 bytes) 000145[0] 1736357952.602496 write 0x571fa8 thru 0x57303e (0x1097 bytes) 000146[1] 1736357952.602918 mapread 0x10933a thru 0x10affb (0x1cc2 bytes) 000147[1] 1736357952.614173 mapwrite 0x899c50 thru 0x89aef4 (0x12a5 bytes) 000150[1] 1736357952.650175 mapwrite 0x4602a8 thru 0x4622a6 (0x1fff bytes) 000151[0] 1736357952.668358 mapread 0x3ee3f9 thru 0x3eed84 (0x98c bytes) 000152[1] 1736357952.673797 trunc from 0x89aef5 to 0x604cb0 (0x296244 bytes) 000153[0] 1736357952.683490 write 0x8aa84b thru 0x8abcd3 (0x1489 bytes) 000155[1] 1736357952.696693 mapwrite 0x3fc2ac thru 0x3fcde2 (0xb37 bytes) 000157[1] 1736357952.724518 mapwrite 0x41eca2 thru 0x420583 (0x18e2 bytes) 000160[1] 1736357952.749305 trunc from 0x8abcd4 to 0x07d00c (0x82ecc7 bytes) 000165[0] 1736357952.770520 mapwrite 0x3a5c86 thru 0x3a69a5 (0xd20 bytes) 000167[1] 1736357952.797770 write 0x1fa0fc thru 0x1fb902 (0x1807 bytes) 000168[0] 1736357952.807728 write 0x114879 thru 0x115735 (0xebd bytes) 000169[1] 1736357952.815882 write 0x3e4c82 thru 0x3e4f3a (0x2b9 bytes) 000170[0] 1736357952.821605 mapread 0x083d70 thru 0x084c9d (0xf2e bytes) 000172[1] 1736357952.830876 mapwrite 0x00c51f thru 0x00ddce (0x18b0 bytes) 000173[0] 1736357952.850748 mapread 0x359a8c thru 0x35b56c (0x1ae1 bytes) 000174[1] 1736357952.861910 read 0x358c94 thru 0x359680 (0x9ed bytes) 000175[0] 1736357952.870120 trunc from 0x3e4f3b to 0x3b32b4 (0x31c86 bytes) 000177[1] 1736357952.877989 mapwrite 0x4fd995 thru 0x4ff920 (0x1f8c bytes) 000182[1] 1736357952.918301 read 0x1edd53 thru 0x1ef61f (0x18cd bytes) 000183[0] 1736357952.929275 write 0x5f2385 thru 0x5f27c6 (0x442 bytes) 000184[0] 1736357952.935982 mapwrite 0x651436 thru 0x652ce8 (0x18b3 bytes) 000185[1] 1736357952.962341 read 0x641df1 thru 0x64395e (0x1b6e bytes) 000186[0] 1736357952.971614 read 0x03ca5b thru 0x03e8a7 (0x1e4d bytes) 000187[0] 1736357953.000704 write 0x1d78b8 thru 0x1d7abc (0x205 bytes) 000188[0] 1736357953.005849 write 0x054ee2 thru 0x054f75 (0x94 bytes) 000189[1] 1736357953.017610 mapread 0x549470 thru 0x549f56 (0xae7 bytes) 000190[1] 1736357953.032933 mapread 0x3602a2 thru 0x3615db (0x133a bytes) 000191[0] 1736357953.038697 read 0x1e0317 thru 0x1e0714 (0x3fe bytes) 000192[0] 1736357953.054305 mapwrite 0x027a27 thru 0x0287a6 (0xd80 bytes) 000194[1] 1736357953.077574 mapwrite 0x98401a thru 0x985e8e (0x1e75 bytes) 000195[0] 1736357953.128769 mapwrite 0x779f89 thru 0x77ae03 (0xe7b bytes) 000196[1] 1736357953.153239 mapread 0x8a5e21 thru 0x8a662f (0x80f bytes) 000198[1] 1736357953.161249 read 0x882241 thru 0x8838c7 (0x1687 bytes) 000200[1] 1736357953.167124 read 0x7e54a8 thru 0x7e71db (0x1d34 bytes) 000202[1] 1736357953.178103 trunc from 0x985e8f to 0x386eab (0x5fefe3 bytes) 000205[0] 1736357953.195169 mapwrite 0x5a6928 thru 0x5a7cec (0x13c5 bytes) 000206[1] 1736357953.227485 write 0x57bc8c thru 0x57d687 (0x19fc bytes) 000207[0] 1736357953.235380 write 0x293102 thru 0x293825 (0x724 bytes) 000209[1] 1736357953.242226 mapwrite 0x2532f6 thru 0x253d2a (0xa35 bytes) 000211[0] 1736357953.265200 mapread 0x4eaa09 thru 0x4ec042 (0x163a bytes) 000213[1] 1736357953.277498 mapwrite 0x7517be thru 0x751da2 (0x5e5 bytes) 000214[0] 1736357953.302393 mapwrite 0x2f839b thru 0x2f88e3 (0x549 bytes) 000218[1] 1736357953.339501 write 0x7ea451 thru 0x7ec297 (0x1e47 bytes) 000220[0] 1736357953.348480 write 0x7cba16 thru 0x7cd656 (0x1c41 bytes) 000221[0] 1736357953.359473 mapread 0x3470f3 thru 0x348784 (0x1692 bytes) 000222[0] 1736357953.361833 trunc from 0x7ec298 to 0x3d6e8d (0x41540a bytes) 000224[0] 1736357953.371896 read 0x1aa9ed thru 0x1abc3e (0x1252 bytes) 000225[1] 1736357953.385713 read 0x11cc9d thru 0x11defe (0x1262 bytes) 000226[1] 1736357953.399017 write 0x1b475f thru 0x1b4ba7 (0x449 bytes) 000229[0] 1736357953.421827 mapread 0x0ff810 thru 0x100e28 (0x1619 bytes) 000230[1] 1736357953.430882 write 0x2ac346 thru 0x2ae23b (0x1ef6 bytes) 000231[0] 1736357953.440860 mapread 0x11681f thru 0x116cf3 (0x4d5 bytes) 000234[1] 1736357953.452827 mapread 0x0bf519 thru 0x0c04ee (0xfd6 bytes) 000235[1] 1736357953.456279 read 0x0ac51a thru 0x0ad30f (0xdf6 bytes) 000236[1] 1736357953.457593 mapwrite 0x1ac4b2 thru 0x1ad7a4 (0x12f3 bytes) 000237[0] 1736357953.472077 write 0x4bb1af thru 0x4bc35a (0x11ac bytes) 000239[0] 1736357953.483296 mapread 0x04b079 thru 0x04b118 (0xa0 bytes) 000240[0] 1736357953.491148 mapread 0x1bad45 thru 0x1bc13e (0x13fa bytes) 000241[0] 1736357953.508470 read 0x3a592a thru 0x3a64e7 (0xbbe bytes) 000245[1] 1736357953.520203 trunc from 0x4bc35b to 0x194e2b (0x32752f bytes) 000246[0] 1736357953.530617 read 0x179b24 thru 0x179b9c (0x79 bytes) 000248[0] 1736357953.539157 read 0x0f66df thru 0x0f67ad (0xcf bytes) 000252[0] 1736357953.545527 trunc from 0x194e2c to 0x0f50c0 (0x9fd6b bytes) 000253[0] 1736357953.553255 write 0x1b1ad7 thru 0x1b2404 (0x92e bytes) 000254[1] 1736357953.557180 trunc from 0x1b2405 to 0x97ab8f (0x7c878b bytes) 000257[1] 1736357953.577273 write 0x8312df thru 0x832df6 (0x1b18 bytes) 000259[0] 1736357953.583490 mapread 0x0af34d thru 0x0af5eb (0x29f bytes) 000260[1] 1736357953.592916 mapwrite 0x9c14ac thru 0x9c318b (0x1ce0 bytes) 000261[0] 1736357953.617160 read 0x32f77c thru 0x330b2d (0x13b2 bytes) 000262[0] 1736357953.631985 mapwrite 0x096595 thru 0x097f00 (0x196c bytes) 000264[0] 1736357953.671309 write 0x935e35 thru 0x9365e3 (0x7af bytes) 000268[1] 1736357953.682224 read 0x07bba5 thru 0x07cfe9 (0x1445 bytes) 000269[1] 1736357953.693521 trunc from 0x9c318c to 0x03add4 (0x9883b7 bytes) 000270[0] 1736357953.707939 read 0x010abd thru 0x011ccb (0x120f bytes) 000272[1] 1736357953.717368 mapwrite 0x71f18c thru 0x720679 (0x14ee bytes) 000273[1] 1736357953.757060 mapwrite 0x2749c9 thru 0x276628 (0x1c60 bytes) 000275[1] 1736357953.773900 write 0x7ac0b4 thru 0x7ace84 (0xdd1 bytes) 000279[0] 1736357953.783787 trunc from 0x7ace85 to 0x70bd29 (0xa115b bytes) 000283[0] 1736357953.807092 write 0x58a026 thru 0x58ab4b (0xb26 bytes) 000284[1] 1736357953.825724 mapread 0x3a925b thru 0x3ab0ff (0x1ea5 bytes) 000285[1] 1736357953.838666 trunc from 0x70bd2a to 0x810f4c (0x105223 bytes) 000286[0] 1736357953.850345 trunc from 0x810f4d to 0x245539 (0x5cba13 bytes) 000290[1] 1736357953.861924 read 0x11d98d thru 0x11e449 (0xabd bytes) 000291[0] 1736357953.871894 mapwrite 0x25bf5a thru 0x25bf91 (0x38 bytes) 000294[0] 1736357953.897496 mapread 0x073809 thru 0x0742cb (0xac3 bytes) 000295[1] 1736357953.900361 write 0x7d83ea thru 0x7d8473 (0x8a bytes) 000297[0] 1736357953.910478 write 0x0e46ae thru 0x0e50f9 (0xa4c bytes) 000298[1] 1736357953.915352 write 0x25d45f thru 0x25e1a9 (0xd4b bytes) 000299[0] 1736357953.932656 mapread 0x5a5b37 thru 0x5a6f9c (0x1466 bytes) 000300[1] 1736357953.951110 mapwrite 0x7c09c0 thru 0x7c254c (0x1b8d bytes) 000302[0] 1736357953.972244 trunc from 0x7d8474 to 0x4aad29 (0x32d74a bytes) 000304[1] 1736357953.988213 mapwrite 0x98f9b2 thru 0x991170 (0x17bf bytes) 000305[0] 1736357954.018691 mapwrite 0x132ad3 thru 0x134164 (0x1692 bytes) 000307[0] 1736357954.045981 read 0x6c23c2 thru 0x6c3912 (0x1551 bytes) 000309[1] 1736357954.064729 trunc from 0x991171 to 0x160e0c (0x830364 bytes) 000310[1] 1736357954.074929 write 0x5317cc thru 0x533050 (0x1885 bytes) 000311[1] 1736357954.076174 mapwrite 0x9d83e2 thru 0x9d9f17 (0x1b36 bytes) 000314[0] 1736357954.099814 write 0x1a4858 thru 0x1a49b1 (0x15a bytes) 000316[0] 1736357954.111620 mapread 0x83c0ae thru 0x83c401 (0x354 bytes) 000318[1] 1736357954.118505 mapwrite 0x34d9ee thru 0x34e74e (0xd61 bytes) 000319[1] 1736357954.137134 mapwrite 0x4714e0 thru 0x471526 (0x47 bytes) 000320[1] 1736357954.152323 mapread 0x87cf74 thru 0x87dd65 (0xdf2 bytes) 000322[1] 1736357954.158357 write 0x3b855b thru 0x3b89fc (0x4a2 bytes) 000323[1] 1736357954.159859 mapread 0x127eca thru 0x128985 (0xabc bytes) 000325[0] 1736357954.168744 write 0x01db73 thru 0x01fb2d (0x1fbb bytes) 000328[0] 1736357954.182798 write 0x424e37 thru 0x426591 (0x175b bytes) 000330[0] 1736357954.193356 write 0x0281fd thru 0x0293f1 (0x11f5 bytes) 000331[0] 1736357954.199747 read 0x8b5655 thru 0x8b74b5 (0x1e61 bytes) 000332[1] 1736357954.207116 mapread 0x957087 thru 0x95897c (0x18f6 bytes) 000333[0] 1736357954.215447 mapread 0x0fd320 thru 0x0fdeb9 (0xb9a bytes) 000335[1] 1736357954.226766 read 0x93fc08 thru 0x941129 (0x1522 bytes) 000336[0] 1736357954.237308 mapwrite 0x30f839 thru 0x30fcf6 (0x4be bytes) 000338[1] 1736357954.265115 mapwrite 0x40e9e9 thru 0x40f219 (0x831 bytes) 000341[1] 1736357954.294917 write 0x216c3e thru 0x218bd5 (0x1f98 bytes) 000343[0] 1736357954.302848 write 0x182f80 thru 0x1836a0 (0x721 bytes) 000347[1] 1736357954.316076 trunc from 0x9d9f18 to 0x748bbf (0x291358 bytes) 000349[0] 1736357954.328389 write 0x8e1d93 thru 0x8e2df0 (0x105e bytes) 000351[1] 1736357954.340718 write 0x32f7f3 thru 0x330b07 (0x1315 bytes) 000352[1] 1736357954.347211 read 0x7d4a11 thru 0x7d4de5 (0x3d5 bytes) 000353[1] 1736357954.355496 mapread 0x6b468d thru 0x6b6451 (0x1dc5 bytes) 000354[1] 1736357954.370082 read 0x458641 thru 0x458784 (0x144 bytes) 000356[1] 1736357954.385770 read 0x5bc9ab thru 0x5bcfb1 (0x607 bytes) 000357[1] 1736357954.386095 read 0x095e6b thru 0x095e78 (0xe bytes) 000358[0] 1736357954.401080 trunc from 0x8e2df1 to 0x99c528 (0xb9738 bytes) 000359[0] 1736357954.421517 write 0x8bba1e thru 0x8bd721 (0x1d04 bytes) 000360[0] 1736357954.425267 mapwrite 0x6119b1 thru 0x612179 (0x7c9 bytes) 000362[1] 1736357954.448267 write 0x40b79f thru 0x40b88e (0xf0 bytes) 000364[0] 1736357954.460579 write 0x4ba2ab thru 0x4bb911 (0x1667 bytes) 000365[0] 1736357954.469624 write 0x5d798a thru 0x5d97fe (0x1e75 bytes) 000366[1] 1736357954.472454 mapread 0x37b84d thru 0x37bb6a (0x31e bytes) 000367[1] 1736357954.484585 read 0x29df0b thru 0x29e173 (0x269 bytes) 000370[1] 1736357954.508611 trunc from 0x99c529 to 0x59cfef (0x3ff539 bytes) 000371[1] 1736357954.525181 trunc from 0x59cff0 to 0x8f8d7e (0x35bd8f bytes) 000372[1] 1736357954.529791 mapwrite 0x3d8ab3 thru 0x3d9283 (0x7d1 bytes) 000373[0] 1736357954.543377 read 0x04695f thru 0x04708c (0x72e bytes) 000375[0] 1736357954.554841 mapwrite 0x375ab9 thru 0x3761a3 (0x6eb bytes) 000376[0] 1736357954.593084 write 0x649f9f thru 0x64b4fb (0x155d bytes) 000377[1] 1736357954.601483 mapwrite 0x940f1c thru 0x942790 (0x1875 bytes) 000378[1] 1736357954.633401 mapwrite 0x132a2f thru 0x133e5a (0x142c bytes) 000379[1] 1736357954.649171 write 0x84de60 thru 0x84f80c (0x19ad bytes) 000382[1] 1736357954.656561 mapread 0x435a82 thru 0x4363df (0x95e bytes) 000383[1] 1736357954.662475 read 0x157a24 thru 0x157cdf (0x2bc bytes) 000385[1] 1736357954.693119 read 0x720caa thru 0x720d07 (0x5e bytes) 000389[0] 1736357954.703371 mapwrite 0x399621 thru 0x39ab0c (0x14ec bytes) 000392[0] 1736357954.741459 mapwrite 0x2e2829 thru 0x2e2885 (0x5d bytes) 000393[0] 1736357954.766368 read 0x5c422c thru 0x5c560a (0x13df bytes) 000394[1] 1736357954.779451 read 0x21ed4c thru 0x21f810 (0xac5 bytes) 000395[1] 1736357954.790196 trunc from 0x942791 to 0x3c7d02 (0x57aa8e bytes) truncating to largest ever: 0x9e31c5 000398[0] 1736357954.807292 trunc from 0x3c7d03 to 0x9e31c4 (0x61b4c2 bytes) 000400[0] 1736357954.815379 trunc from 0x9e31c5 to 0x39638c (0x64ce38 bytes) 000401[0] 1736357954.821998 mapread 0x2f4e67 thru 0x2f6784 (0x191e bytes) 000402[1] 1736357954.827875 write 0x6fa6f2 thru 0x6fbc58 (0x1567 bytes) 000403[0] 1736357954.832971 read 0x1428bb thru 0x1445a2 (0x1ce8 bytes) 000407[0] 1736357954.851768 write 0x87ff4a thru 0x880fbb (0x1072 bytes) 000408[1] 1736357954.855689 write 0x347d2b thru 0x3484d2 (0x7a8 bytes) 000409[1] 1736357954.867515 mapwrite 0x2407ec thru 0x240fdf (0x7f4 bytes) 000412[0] 1736357954.909877 trunc from 0x880fbc to 0x40c6ea (0x4748d1 bytes) 000413[0] 1736357954.931456 write 0x1d6b73 thru 0x1d89da (0x1e68 bytes) 000414[1] 1736357954.947844 write 0x49cbf5 thru 0x49e1d6 (0x15e2 bytes) 000415[1] 1736357954.956541 trunc from 0x49e1d7 to 0x5b07de (0x112608 bytes) 000416[1] 1736357954.971372 mapread 0x3c1518 thru 0x3c24bd (0xfa6 bytes) 000419[1] 1736357954.984637 write 0x9cdf16 thru 0x9cfad5 (0x1bc0 bytes) 000420[1] 1736357954.985845 mapread 0x3c8e17 thru 0x3c8ff6 (0x1e0 bytes) 000421[0] 1736357954.992697 mapread 0x49479f thru 0x496078 (0x18da bytes) 000422[0] 1736357955.008235 mapread 0x2fe77b thru 0x2ff39f (0xc25 bytes) 000423[0] 1736357955.017108 mapwrite 0x0b27bb thru 0x0b2cd0 (0x516 bytes) 000425[1] 1736357955.042246 mapread 0x9898b7 thru 0x98a1ba (0x904 bytes) 000427[1] 1736357955.050722 trunc from 0x9cfad6 to 0x8c9fcd (0x105b08 bytes) 000428[1] 1736357955.060531 trunc from 0x8c9fce to 0x82fbaa (0x9a423 bytes) 000429[0] 1736357955.064835 read 0x0f8d88 thru 0x0f9079 (0x2f2 bytes) 000431[1] 1736357955.069779 write 0x28788b thru 0x288bd5 (0x134b bytes) 000432[0] 1736357955.073870 mapwrite 0x3e9f21 thru 0x3ea56c (0x64c bytes) 000433[1] 1736357955.104360 read 0x81fbd0 thru 0x82023e (0x66f bytes) 000434[1] 1736357955.111131 write 0x86b97d thru 0x86d2db (0x195f bytes) 000435[1] 1736357955.116899 mapwrite 0x2fdee8 thru 0x2ff52b (0x1644 bytes) 000436[1] 1736357955.145337 mapread 0x639162 thru 0x63a3b4 (0x1253 bytes) 000437[1] 1736357955.152001 write 0x315f13 thru 0x3175ea (0x16d8 bytes) 000440[1] 1736357955.165970 read 0x3bf5b0 thru 0x3c149e (0x1eef bytes) 000441[0] 1736357955.183357 write 0x3e7a2e thru 0x3e990a (0x1edd bytes) 000442[1] 1736357955.195742 mapread 0x614013 thru 0x6140ec (0xda bytes) 000443[0] 1736357955.201365 mapread 0x02909d thru 0x029fb4 (0xf18 bytes) 000444[1] 1736357955.236738 mapwrite 0x132a8c thru 0x1330d5 (0x64a bytes) 000445[1] 1736357955.262496 read 0x6c7886 thru 0x6c7a00 (0x17b bytes) 000446[1] 1736357955.273094 trunc from 0x86d2dc to 0x61b5fb (0x251ce0 bytes) 000448[0] 1736357955.283870 mapwrite 0x3ada8f thru 0x3ae081 (0x5f3 bytes) 000449[0] 1736357955.305296 trunc from 0x61b5fc to 0x2e7047 (0x3345b4 bytes) 000450[0] 1736357955.311476 mapread 0x11fa5c thru 0x1209ec (0xf91 bytes) 000451[0] 1736357955.318759 trunc from 0x2e7048 to 0x1ef5ad (0xf7a9a bytes) 000452[0] 1736357955.323118 write 0x267f12 thru 0x269282 (0x1371 bytes) 000455[0] 1736357955.332173 trunc from 0x269283 to 0x5074aa (0x29e228 bytes) 000459[0] 1736357955.341847 mapread 0x0e2c08 thru 0x0e33d6 (0x7cf bytes) 000461[1] 1736357955.350920 mapwrite 0x86306d thru 0x863879 (0x80d bytes) 000462[1] 1736357955.372832 write 0x7e13f0 thru 0x7e2c9a (0x18ab bytes) 000463[0] 1736357955.375179 mapread 0x3e660e thru 0x3e75b9 (0xfac bytes) 000464[1] 1736357955.385667 mapread 0x57f63b thru 0x5803cc (0xd92 bytes) 000465[0] 1736357955.389008 trunc from 0x86387a to 0x6f4383 (0x16f4f6 bytes) 000466[0] 1736357955.397146 mapread 0x017038 thru 0x017501 (0x4ca bytes) 000467[0] 1736357955.402261 trunc from 0x6f4384 to 0x2b0176 (0x44420d bytes) 000469[1] 1736357955.406169 mapread 0x026c3d thru 0x02780f (0xbd3 bytes) 000471[0] 1736357955.416538 read 0x0817c2 thru 0x081b58 (0x397 bytes) 000473[0] 1736357955.427123 mapwrite 0x3bd2ad thru 0x3bf282 (0x1fd6 bytes) 000474[0] 1736357955.459161 mapwrite 0x9d9ac1 thru 0x9da379 (0x8b9 bytes) 000475[0] 1736357955.481430 write 0x72ec65 thru 0x72edc5 (0x161 bytes) 000476[0] 1736357955.483083 read 0x434a35 thru 0x435054 (0x620 bytes) 000479[1] 1736357955.501160 mapread 0x155819 thru 0x156bf6 (0x13de bytes) 000481[0] 1736357955.514332 mapwrite 0x3d23b4 thru 0x3d241c (0x69 bytes) 000482[1] 1736357955.529547 read 0x8a1aa2 thru 0x8a1f19 (0x478 bytes) 000483[0] 1736357955.540146 trunc from 0x9da37a to 0x1091d6 (0x8d11a3 bytes) 000484[1] 1736357955.548291 write 0x61478e thru 0x614d19 (0x58c bytes) 000485[1] 1736357955.554731 mapwrite 0x3663be thru 0x367388 (0xfcb bytes) 000489[1] 1736357955.591327 write 0x1fc9c1 thru 0x1fd801 (0xe41 bytes) 000493[1] 1736357955.603234 trunc from 0x614d1a to 0x877f19 (0x263200 bytes) 000496[0] 1736357955.612202 trunc from 0x877f1a to 0x6e6368 (0x191bb1 bytes) 000499[1] 1736357955.631635 trunc from 0x6e6369 to 0x58390f (0x162a59 bytes) 000500[1] 1736357955.646413 read 0x2b129d thru 0x2b2d86 (0x1aea bytes) All operations completed A-OK! PASS 16b (6s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 12:39:16 (1736357956) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 12:39:17 (1736357957) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0673631 s, 31.1 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0527059 s, 39.8 MB/s ldlm.namespaces.MGC192.168.203.124@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff88012a60b800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8801369f4000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff88012a60b800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8801369f4000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff88012a60b800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8801369f4000.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.50737 s, 41.8 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.557801 s, 188 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.558464 s, 188 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.18217 s, 88.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.64221 s, 63.9 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.922849 s, 114 MB/s PASS 16d (10s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 12:39:27 (1736357967) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0616519 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0296743 s, 35.3 MB/s PASS 16e (2s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 12:39:29 (1736357969) TIMEOUT PASS 16f (21s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 12:39:50 (1736357990) TIMEOUT PASS 16g (22s) == sanityn test 16i: read after truncate file ============ 12:40:12 (1736358012) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (1s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 12:40:13 (1736358013) fail_loc=0x8000030a PASS 17 (4s) == sanityn test 18: mmap sanity check =========================================================================================== 12:40:17 (1736358017) mmap test1: basic mmap operation (PASS, 0.011518s) mmap test2: MAP_PRIVATE not write back (PASS, 0.008887s) mmap test3: concurrent mmap ops on two nodes (PASS, 2.8861s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.057s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.021127s) mmap test6: check mmap write/read content on two nodes (PASS, 0.027052s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.012s) PASS 18 (8s) == sanityn test 19: test concurrent uncached read races ========================================================================= 12:40:25 (1736358025) oleg324-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg324-client: oleg324-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 12:40:26 (1736358026) PASS 20 (2s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 12:40:28 (1736358028) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (1s) == sanityn test 23: others should see updated atime while another read============================================================== 12:40:29 (1736358029) now is 1736358030 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6858 new atime is 1736358091 PASS 23 (63s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 12:41:32 (1736358092) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 374 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 375 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 374 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 375 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 8.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 5.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 13.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117974 374 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118071 375 117696 1% /mnt/lustre2[OST:1] filesystem_summary: 235665 369 235296 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 374 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 375 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 117974 374 117600 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118071 375 117696 1% /mnt/lustre[OST:1] filesystem_summary: 235665 369 235296 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117974 374 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118071 375 117696 1% /mnt/lustre2[OST:1] filesystem_summary: 235665 369 235296 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre2 PASS 24a (1s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 12:41:33 (1736358093) PASS 24b (2s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 12:41:35 (1736358095) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (1s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 12:41:36 (1736358096) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 12:41:37 (1736358097) utime: good mknod times 1736358096 <= 1736358097 <= 1736358097 for /mnt/lustre/f26a utime: good mknod times 1736358096 <= 1736358097 <= 1736358097 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (2s) == sanityn test 26b: sync mtime between ost and mds ====== 12:41:39 (1736358099) PASS 26b (4s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 12:41:43 (1736358103) 0 PASS 30 (4s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 12:41:47 (1736358107) fail_loc=0x314 PASS 31a (2s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 12:41:49 (1736358109) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg324-server mds-ost sync done. sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.203.124@tcp:/lustre 7542784 13312 7523328 1% /mnt/lustre2 PASS 31b (18s) == sanityn test 31r: open-rename(replace) race =========== 12:42:07 (1736358127) -rw-r--r-- 1 root root 0 Jan 8 12:42 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (5s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 12:42:12 (1736358132) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 12:42:13 (1736358133) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (0s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 12:42:13 (1736358133) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 12:42:14 (1736358134) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 12:42:15 (1736358135) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg324-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88012a60b800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8801369f4000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff88012a60b800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8801369f4000.ost_server_uuid in FULL state after 0 sec oleg324-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff88012a60b800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8801369f4000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff88012a60b800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8801369f4000.ost_server_uuid in FULL state after 0 sec oleg324-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88012a60b800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8801369f4000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff88012a60b800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8801369f4000.ost_server_uuid in FULL state after 0 sec oleg324-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff88012a60b800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8801369f4000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff88012a60b800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8801369f4000.ost_server_uuid in IDLE state after 0 sec oleg324-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff88012a60b800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8801369f4000.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff88012a60b800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8801369f4000.ost_server_uuid in FULL state after 0 sec oleg324-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff88012a60b800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8801369f4000.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff88012a60b800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8801369f4000.ost_server_uuid in IDLE state after 0 sec PASS 34 (70s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 12:43:25 (1736358205) fail_loc=0x80000317 Race attempt 0 Wait for 15539 15679 for 60 sec... - open/close 3987 (time 1736358217.51 total 10.00 last 398.66) total: 4000 open/close in 10.02 seconds: 399.04 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.203.124@tcp:/lustre 7.2G 15M 7.2G 1% /mnt/lustre 192.168.203.124@tcp:/lustre 7.2G 15M 7.2G 1% /mnt/lustre2 PASS 35 (64s) == sanityn test 36: handle ESTALE/open-unlink correctly == 12:44:29 (1736358269) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg324-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.26059 s, 201 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6858 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.290195 s, 181 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6858 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.305201 s, 172 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6858 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.326006 s, 161 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6858 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.349059 s, 150 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6858 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.322865 s, 162 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6858 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.307985 s, 170 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6858 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.286811 s, 183 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6858 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.274353 s, 191 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6858 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.286091 s, 183 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6858 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.278249 s, 188 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6858 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7523328) after_dd(7472128) after(7523328) PASS 36 (228s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 12:48:17 (1736358497) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6858 total: 10000 create in 9.39 seconds: 1064.74 ops/second PASS 37 (12s) == sanityn test 39a: test from 11063 ============================================================================================ 12:48:29 (1736358509) repeat after cancel_lru_locks PASS 39a (1s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 12:48:30 (1736358510) repeat after cancel_lru_locks PASS 39b (3s) == sanityn test 39c: check truncate mtime update ================================================================================ 12:48:33 (1736358513) repeat after cancel_lru_locks PASS 39c (2s) == sanityn test 39d: sync write should update mtime ====== 12:48:35 (1736358515) fail_loc=0x411 fail_loc=0 PASS 39d (1s) == sanityn test 40a: pdirops: create vs others ======================================================================== 12:48:37 (1736358517) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (6s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 12:48:44 (1736358524) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (7s) == sanityn test 40c: pdirops: link and others ======================================================================== 12:48:50 (1736358530) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (7s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 12:48:57 (1736358537) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (7s) == sanityn test 40e: pdirops: rename and others ======================================================================== 12:49:04 (1736358544) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (7s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 12:49:11 (1736358551) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (4s) == sanityn test 41b: pdirops: create vs create ======================================================================== 12:49:15 (1736358555) open: File exists Conflict PASS 41b (4s) == sanityn test 41c: pdirops: create vs link ======================================================================== 12:49:19 (1736358559) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (4s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 12:49:23 (1736358563) Conflict PASS 41d (4s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 12:49:27 (1736358567) Conflict PASS 41e (4s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 12:49:32 (1736358572) Conflict PASS 41f (5s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 12:49:36 (1736358576) Conflict PASS 41g (4s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 12:49:40 (1736358580) Conflict PASS 41h (4s) == sanityn test 41i: reint_open: create vs create ======== 12:49:44 (1736358584) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (643s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 13:00:27 (1736359227) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (4s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 13:00:31 (1736359231) open: File exists Conflict PASS 42b (4s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 13:00:35 (1736359235) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (4s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 13:00:39 (1736359239) Conflict PASS 42d (4s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 13:00:43 (1736359243) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (5s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 13:00:48 (1736359248) Conflict PASS 42f (4s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 13:00:52 (1736359252) Conflict PASS 42g (4s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 13:00:56 (1736359256) Conflict PASS 42h (4s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 13:01:00 (1736359260) PASS 43a (10s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 13:01:10 (1736359270) Conflict PASS 43b (4s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 13:01:14 (1736359274) Conflict PASS 43c (4s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 13:01:18 (1736359278) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (4s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 13:01:22 (1736359282) Conflict PASS 43e (4s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 13:01:26 (1736359286) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (5s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 13:01:31 (1736359291) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (4s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 13:01:35 (1736359295) Conflict PASS 43h (4s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 13:01:39 (1736359299) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 13:01:40 (1736359300) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (31s) == sanityn test 43k: unlink vs create ==================== 13:02:12 (1736359332) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (348s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 13:07:59 (1736359679) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Wed Jan 8 13:08:02 EST 2025 PASS 44a (4s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 13:08:03 (1736359683) open: File exists Conflict PASS 44b (4s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 13:08:07 (1736359687) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (4s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 13:08:11 (1736359691) Conflict PASS 44d (4s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 13:08:15 (1736359695) Conflict PASS 44e (4s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 13:08:19 (1736359699) Conflict PASS 44f (4s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 13:08:23 (1736359703) Conflict PASS 44g (5s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 13:08:28 (1736359708) Conflict PASS 44h (4s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 13:08:32 (1736359712) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (0s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 13:08:33 (1736359713) PASS 45a (19s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 13:08:52 (1736359732) Conflict PASS 45b (4s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 13:08:56 (1736359736) Conflict PASS 45c (4s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 13:09:00 (1736359740) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (5s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 13:09:05 (1736359745) Conflict PASS 45e (4s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 13:09:09 (1736359749) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (4s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 13:09:13 (1736359753) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (4s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 13:09:17 (1736359757) Conflict PASS 45h (3s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 13:09:21 (1736359761) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (0s) == sanityn test 45j: read vs rename ====================== 13:09:21 (1736359761) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (353s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 13:15:14 (1736360114) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (4s) == sanityn test 46b: pdirops: link vs create ======================================================================== 13:15:18 (1736360118) open: File exists Conflict PASS 46b (4s) == sanityn test 46c: pdirops: link vs link ======================================================================== 13:15:22 (1736360122) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (5s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 13:15:27 (1736360127) Conflict PASS 46d (4s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 13:15:31 (1736360131) Conflict PASS 46e (4s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 13:15:35 (1736360135) Conflict PASS 46f (4s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 13:15:39 (1736360139) Conflict PASS 46g (4s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 13:15:43 (1736360143) Conflict PASS 46h (5s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 13:15:48 (1736360148) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (0s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 13:15:48 (1736360148) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 13:15:49 (1736360149) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 13:15:50 (1736360150) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (0s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 13:15:50 (1736360150) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 13:15:51 (1736360151) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 13:15:52 (1736360152) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 13:15:53 (1736360153) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (0s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 13:15:53 (1736360153) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.00454046 s, 2.3 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 13:16:00 (1736360160) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00412461 s, 38.5 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (24474) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 13:16:04 (1736360164) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00328561 s, 312 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126865 s, 82.7 MB/s 3145728 PASS 51b (13s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 13:16:17 (1736360177) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00352574 s, 290 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 13:16:23 (1736360183) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00746518 s, 140 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (2s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 13:16:25 (1736360185) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2435 0x983 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2435 0x983 0 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 13:16:29 (1736360189) llite.lustre-ffff88012a60b800.xattr_cache=1 llite.lustre-ffff8801369f4000.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504963 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-08 13:16:29.000000000 -0500 Modify: 2025-01-08 13:16:29.000000000 -0500 Change: 2025-01-08 13:16:29.000000000 -0500 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504966 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-08 13:16:35.000000000 -0500 Modify: 2025-01-08 13:16:35.000000000 -0500 Change: 2025-01-08 13:16:35.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504969 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-08 13:16:40.000000000 -0500 Modify: 2025-01-08 13:16:40.000000000 -0500 Change: 2025-01-08 13:16:40.000000000 -0500 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-08 13:16:40.000000000 -0500 Modify: 2025-01-08 13:16:40.000000000 -0500 Change: 2025-01-08 13:16:40.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-08 13:16:45.000000000 -0500 Modify: 2025-01-08 13:16:46.000000000 -0500 Change: 2025-01-08 13:16:46.000000000 -0500 Birth: - File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504972 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-08 13:16:45.000000000 -0500 Modify: 2025-01-08 13:16:46.000000000 -0500 Change: 2025-01-08 13:16:46.000000000 -0500 Birth: - PASS 54 (23s) == sanityn test 55a: rename vs unlink target dir ========= 13:16:52 (1736360212) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 13:16:59 (1736360219) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (6s) == sanityn test 55c: rename vs unlink orphan target dir == 13:17:06 (1736360226) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 13:17:17 (1736360237) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) == sanityn test 60: Verify data_version behaviour ======== 13:17:26 (1736360246) PASS 60 (2s) == sanityn test 70a: cd directory && rm directory ======== 13:17:28 (1736360248) PASS 70a (1s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 13:17:29 (1736360249) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 13:17:31 (1736360251) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00289541 s, 22.6 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 13:17:32 (1736360252) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00386746 s, 16.9 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 13:17:33 (1736360253) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (0s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 13:17:33 (1736360253) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00369267 s, 17.7 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 13:17:34 (1736360254) llite.lustre-ffff88012a60b800.xattr_cache=1 llite.lustre-ffff8801369f4000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 13:17:36 (1736360256) llite.lustre-ffff88012a60b800.xattr_cache=1 llite.lustre-ffff8801369f4000.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (1s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 13:17:37 (1736360257) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00266346 s, 384 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00484091 s, 212 kB/s 3120: get lock2 3121: get lock1 3121: done 3121: get lock2 3120: done 3120: get lock1 fcntl cmd 7 failed: Resource deadlock avoided 3120: cannot get lock1: Resource deadlock avoided fcntl cmd 7 failed: Resource deadlock avoided 3121: cannot get lock2: Resource deadlock avoided 3121: exit rc=0 3120: done 3120: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 13:17:43 (1736360263) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.014394 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00687327 s, 153 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 3755 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (6s) == sanityn test 76: Verify MDT open_files listing ======== 13:17:49 (1736360269) opening files: ........ closing files: ........ PASS 76 (14s) == sanityn test 77a: check FIFO NRS policy =============== 13:18:03 (1736360283) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.115346 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008797 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122944 s, 85.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214032 s, 49.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030544 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231813 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276287 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265682 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253001 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0348863 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235091 s, 44.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280243 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254703 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307102 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295491 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270502 s, 38.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a60b800.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff88012a60b800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0520245 s, 20.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a60b800.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102448 s, 102 MB/s 1048576 bytes (1.0 MB) copied, 0.0103617 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00191494 s, 548 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00692785 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00436298 s, 240 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00627568 s, 167 MB/s 1048576 bytes (1.0 MB) copied, 0.00811168 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00178295 s, 588 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00210775 s, 497 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00751623 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00235912 s, 444 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00940276 s, 112 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00442937 s, 237 MB/s , 0.00189642 s, 553 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00130247 s, 805 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00137166 s, 764 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg324-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) == sanityn test 77b: check CRR-N NRS policy ============== 13:18:05 (1736360285) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0951836 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110269 s, 95.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00790378 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108979 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194451 s, 53.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918389 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275153 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328818 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193066 s, 54.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316512 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01901 s, 55.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318201 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260434 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0419617 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174316 s, 60.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0301955 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361662 s, 29.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a60b800.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff88012a60b800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008187 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00807102 s, 130 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00987699 s, 106 MB/s , 0.00897429 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00826953 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00846128 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972373 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00919655 s, 114 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00136289 s, 769 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00887529 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108353 s, 96.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948499 s, 111 MB/s 1048576 bytes (1.0 MB) copied, 0.00304395 s, 344 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00975082 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00953628 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00979835 s, 107 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg324-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.101867 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00686515 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0079253 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215246 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250562 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216468 s, 48.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216462 s, 48.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275486 s, 38.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02515 s, 41.7 MB/s , 0.0212029 s, 49.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226752 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227443 s, 46.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0322902 s, 32.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0132525 s, 79.1 MB/s , 0.0346936 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265232 s, 39.5 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012a60b800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110334 s, 95.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0082971 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00145748 s, 719 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00388538 s, 270 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00798489 s, 131 MB/s 1048576 bytes (1.0 MB) copied, 0.0091178 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0041088 s, 255 MB/s , 0.00170243 s, 616 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104741 s, 100 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00840071 s, 125 MB/s , 0.00899044 s, 117 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00579694 s, 181 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00669364 s, 157 MB/s 1048576 bytes (1.0 MB) copied, 0.0079063 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00204506 s, 513 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00139833 s, 750 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00187549 s, 559 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a60b800.lock_unused_count=1 removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg324-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) == sanityn test 77c: check ORR NRS policy ================ 13:18:08 (1736360288) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.110761 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00720726 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176043 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233139 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253906 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216888 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226334 s, 46.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232208 s, 45.2 MB/s 1048576 bytes (1.0 MB) copied, 0.02517 s, 41.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0231859 s, 45.2 MB/s , 0.0184588 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185453 s, 56.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0337007 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.028554 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212084 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227983 s, 46.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a60b800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88012a60b800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145019 s, 72.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116648 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00432352 s, 243 MB/s , 0.00778437 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00723497 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00580478 s, 181 MB/s 1048576 bytes (1.0 MB) copied, 0.00764382 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00615894 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0150901 s, 69.5 MB/s 1048576 bytes (1.0 MB) copied, 0.00750953 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00804064 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00877229 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918749 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00954607 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108776 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108729 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112084 s, 93.6 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg324-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.133632 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00933842 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00975951 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100448 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0264309 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192269 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118097 s, 88.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288044 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0312885 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0358524 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0299418 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342375 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294772 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352525 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457233 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335209 s, 31.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a60b800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff88012a60b800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0379897 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0056964 s, 184 MB/s , 0.00951961 s, 110 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00756344 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948402 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00707295 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00769868 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00409728 s, 256 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00950144 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00945253 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0099529 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00340622 s, 308 MB/s , 0.00771017 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00919707 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114363 s, 91.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106277 s, 98.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00125928 s, 833 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a60b800.lock_unused_count=1 removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg324-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0872945 s, 192 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00910772 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010836 s, 96.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130756 s, 80.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113048 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00974424 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025102 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107261 s, 97.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155002 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227549 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0323538 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152082 s, 68.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0362791 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392411 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0377847 s, 27.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a60b800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88012a60b800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365315 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0435693 s, 24.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012a60b800.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00788704 s, 133 MB/s , 0.0078481 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00741567 s, 141 MB/s 1048576 bytes (1.0 MB) copied, 0.00942094 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00714298 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112852 s, 92.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102945 s, 102 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00877529 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00714182 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00933294 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00871149 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100659 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00478068 s, 219 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00899395 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00118896 s, 882 MB/s , 0.00921428 s, 114 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg324-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (4s) == sanityn test 77d: check TRR nrs policy ================ 13:18:12 (1736360292) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.113455 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101937 s, 103 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00855481 s, 123 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107351 s, 97.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0141513 s, 74.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0175738 s, 59.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195743 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00841365 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152447 s, 68.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203714 s, 51.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185766 s, 56.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0344906 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237054 s, 44.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326957 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211856 s, 49.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271798 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263812 s, 39.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a60b800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff88012a60b800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00719908 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00897162 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00158909 s, 660 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104658 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00860687 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00897213 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.001946 s, 539 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948688 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111767 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103833 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010895 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00908788 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130449 s, 80.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012165 s, 86.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123967 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134038 s, 78.2 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg324-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0901543 s, 186 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0079458 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137765 s, 76.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139035 s, 75.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00936092 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166339 s, 63.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0221652 s, 47.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234862 s, 44.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180864 s, 58.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121538 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027116 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143793 s, 72.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169124 s, 62.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281985 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161113 s, 65.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274235 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copiedldlm.namespaces.lustre-OST0000-osc-ffff88012a60b800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88012a60b800.lock_unused_count=1 , 0.0250253 s, 41.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00797997 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00205894 s, 509 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0015382 s, 682 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00827513 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00915616 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00436471 s, 240 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00137708 s, 761 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00933547 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00938008 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00696566 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00411649 s, 255 MB/s , 0.00153182 s, 685 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00887222 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00674117 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105155 s, 99.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00144 s, 728 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg324-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0923469 s, 182 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00964922 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108349 s, 96.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144901 s, 72.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253209 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207199 s, 50.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239257 s, 43.8 MB/s 1048576 bytes (1.0 MB) copied, 0.00815563 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263788 s, 39.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219553 s, 47.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289892 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268234 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248486 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251942 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300184 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210061 s, 49.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00966688 s, 108 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a60b800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88012a60b800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00711808 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00646527 s, 162 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00702087 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00676259 s, 155 MB/s , 0.00979839 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112934 s, 92.8 MB/s , 0.00931647 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00446882 s, 235 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00985938 s, 106 MB/s 1048576 bytes (1.0 MB) copied, 0.010682 s, 98.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00989342 s, 106 MB/s 1048576 bytes (1.0 MB) copied, 0.0130677 s, 80.2 MB/s 1048576 bytes (1.0 MB) copied, 0.00440904 s, 238 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012938 s, 81.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00142003 s, 738 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00468701 s, 224 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg324-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (3s) == sanityn test 77e: check TBF NID nrs policy ============ 13:18:15 (1736360295) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.203.24@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.082216 s, 204 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00800285 s, 131 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0146601 s, 71.5 MB/s , 0.0149637 s, 70.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015688 s, 66.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00834556 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0223311 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234445 s, 44.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0303827 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319474 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199717 s, 52.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260786 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0377861 s, 27.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0216023 s, 48.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189718 s, 55.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291565 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012103 s, 86.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00788394 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00718972 s, 146 MB/s , 0.00714529 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00725294 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00841666 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0408317 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0427803 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00959616 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102315 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0431407 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131386 s, 79.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464722 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117436 s, 89.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114107 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119691 s, 87.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010725 s, 97.8 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg324-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0919187 s, 183 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0073299 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136698 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280576 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275061 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289337 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255011 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225673 s, 46.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259983 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034406 s, 30.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0375645 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0399812 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0368631 s, 28.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0409289 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0403634 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0393494 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0369475 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00559643 s, 187 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00880835 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00688589 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00725498 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00910809 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00858721 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00809102 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00767105 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918478 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00890188 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00841163 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00920764 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0088042 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00874035 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00883766 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102252 s, 103 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg324-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.102115 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00813277 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011265 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00957233 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208653 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137038 s, 76.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280699 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205533 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114379 s, 91.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154295 s, 68.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027111 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0341351 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199057 s, 52.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335547 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118062 s, 88.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140371 s, 74.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117381 s, 89.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a60b800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff88012a60b800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00931441 s, 113 MB/s 1048576 bytes (1.0 MB) copied, 0.00662507 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00964757 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102675 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00192414 s, 545 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0128617 s, 81.5 MB/s 1048576 bytes (1.0 MB) copied, 0.00917959 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113138 s, 92.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0106956 s, 98.0 MB/s , 0.0102166 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110933 s, 94.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00429924 s, 244 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copiedldlm.namespaces.lustre-OST0000-osc-ffff88012a60b800.lock_unused_count=1 , 0.0121995 s, 86.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131192 s, 79.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127376 s, 82.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00148513 s, 706 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg324-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.105765 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00981552 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139854 s, 75.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140043 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239204 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140892 s, 74.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270813 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176013 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217602 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219729 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277159 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0334535 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0452996 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0404319 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0539549 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0408472 s, 25.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012a60b800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.057895 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00670878 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00669839 s, 157 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00369883 s, 283 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00892215 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00853239 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107393 s, 97.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00820317 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00678769 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00811947 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00370778 s, 283 MB/s 1+0 records in 1+0 records out ldlm.namespaces.lustre-OST0000-osc-ffff88012a60b800.lock_unused_count=2 1048576 bytes (1.0 MB) copied, 0.00179771 s, 583 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00257807 s, 407 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00164478 s, 638 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00530269 s, 198 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00120156 s, 873 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00144023 s, 728 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg324-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (6s) == sanityn test 77f: check TBF JobID nrs policy ========== 13:18:21 (1736360301) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg324-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.106326 s, 158 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00819747 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115387 s, 90.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161526 s, 64.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0190266 s, 55.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0168873 s, 62.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254879 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0363865 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0369648 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0355069 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0373369 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.034374 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345843 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0339662 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0434542 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0441516 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0541146 s, 19.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00816599 s, 128 MB/s 1048576 bytes (1.0 MB) copied, 0.00931137 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00211989 s, 495 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117299 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00863515 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101444 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983302 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011508 s, 91.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0098757 s, 106 MB/s 1048576 bytes (1.0 MB) copied, 0.0105751 s, 99.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118366 s, 88.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012544 s, 83.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118796 s, 88.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116438 s, 90.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117288 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133472 s, 78.6 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg324-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg324-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.108206 s, 155 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0095331 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00982294 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109831 s, 95.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00870277 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00826467 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306158 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161756 s, 64.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188717 s, 55.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145425 s, 72.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332102 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390474 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.041533 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0198617 s, 52.8 MB/s , 0.0422715 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219947 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139998 s, 74.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00188216 s, 557 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00819879 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103615 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00861086 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00940548 s, 111 MB/s , 0.00760322 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00941085 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012757 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110163 s, 95.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113026 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106446 s, 98.5 MB/s , 0.00759839 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108517 s, 96.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0126565 s, 82.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0126985 s, 82.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013788 s, 76.0 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg324-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg324-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.121299 s, 138 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137121 s, 76.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203832 s, 51.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290779 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0302032 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338356 s, 31.0 MB/s , 0.0276311 s, 37.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232196 s, 45.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0248955 s, 42.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0373141 s, 28.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0297179 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0432675 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0287565 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326955 s, 32.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390206 s, 26.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0428181 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0313534 s, 33.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a60b800.lock_unused_count=1 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00786337 s, 133 MB/s 1+0 records in 1+0 records out , 0.00192842 s, 544 MB/s 1048576 bytes (1.0 MB) copied, 0.00988323 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00830034 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105142 s, 99.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00812571 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00760395 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109238 s, 96.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00913006 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00923849 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0097907 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109187 s, 96.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111747 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109467 s, 95.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116906 s, 89.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115187 s, 91.0 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg324-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg324-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.129315 s, 130 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141113 s, 74.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014262 s, 73.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255919 s, 41.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288057 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236241 s, 44.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0335753 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0299818 s, 35.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.024073 s, 43.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0380571 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0412485 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267198 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215462 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316507 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256836 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229573 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308674 s, 34.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00612554 s, 171 MB/s , 0.00973399 s, 108 MB/s , 0.0068109 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00886162 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00851377 s, 123 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00614594 s, 171 MB/s , 0.00657637 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00158474 s, 662 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120997 s, 86.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00174167 s, 602 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00175677 s, 597 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00174437 s, 601 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124881 s, 84.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00164971 s, 636 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00259085 s, 405 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00173797 s, 603 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a60b800.lock_unused_count=1 removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg324-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (6s) == sanityn test 77g: Change TBF type directly ============ 13:18:27 (1736360307) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (3s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 13:18:30 (1736360310) oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg324-client: oleg324-server: ssh exited with exit code 2 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg324-client: oleg324-server: ssh exited with exit code 12 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg324-client: oleg324-server: ssh exited with exit code 12 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg324-client: oleg324-server: ssh exited with exit code 12 PASS 77h (2s) == sanityn test 77i: Change rank of TBF rule ============= 13:18:32 (1736360312) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (6s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 13:18:38 (1736360318) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0999686 s, 168 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848529 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013401 s, 78.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122817 s, 85.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149161 s, 70.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161609 s, 64.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00702316 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202067 s, 51.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116073 s, 90.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112458 s, 93.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291079 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331574 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025767 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238453 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267525 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00971293 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02401 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00671938 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0094227 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00693646 s, 151 MB/s 1048576 bytes (1.0 MB) copied, 0.00631532 s, 166 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117698 s, 89.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00937937 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00851744 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00782212 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00998106 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0100571 s, 104 MB/s 1048576 bytes (1.0 MB) copied, 0.00991797 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100601 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0094735 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100628 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102408 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102314 s, 102 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg324-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92828 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4265 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (46s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 13:19:24 (1736360364) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg324-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.637808 s, 26.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00958526 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158014 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174314 s, 60.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020271 s, 51.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149581 s, 70.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103923 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101435 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179172 s, 58.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.372459 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.367426 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.375759 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.374301 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.39355 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.388241 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.391334 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.398287 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15772 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155406 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155523 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155877 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156708 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.15687 s, 6.7 MB/s , 0.15636 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158762 s, 6.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158309 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157004 s, 6.7 MB/s 1048576 bytes (1.0 MB) copied, 0.158833 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157534 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.158621 s, 6.6 MB/s , 0.15876 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159277 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.161775 s, 6.5 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg324-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8262 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9731 s, 5.2 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg324-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.642001 s, 26.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.165155 s, 6.3 MB/s , 0.166393 s, 6.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.163069 s, 6.4 MB/s , 0.169437 s, 6.2 MB/s , 0.166909 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.165732 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.183024 s, 5.7 MB/s 1048576 bytes (1.0 MB) copied, 0.18161 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.366566 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.375542 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.370751 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.378845 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.381464 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.384777 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.392341 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.395981 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.151809 s, 6.9 MB/s , 0.151571 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152399 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152424 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155295 s, 6.8 MB/s , 0.153098 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155637 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154439 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154722 s, 6.8 MB/s , 0.155265 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154821 s, 6.8 MB/s 1048576 bytes (1.0 MB) copied, 0.156787 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156308 s, 6.7 MB/s 1048576 bytes (1.0 MB) copied, 0.15766 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15599 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156747 s, 6.7 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg324-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8263 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9723 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (135s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 13:21:39 (1736360499) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg324-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.200413 s, 83.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166978 s, 62.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182521 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207682 s, 50.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165393 s, 63.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246107 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193426 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017616 s, 59.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229832 s, 45.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0674368 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.069146 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0671304 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0813279 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0931362 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0830287 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.096654 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0745218 s, 14.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0071615 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00814489 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00591103 s, 177 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00638852 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100818 s, 104 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.011756 s, 89.2 MB/s , 0.00765242 s, 137 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0088868 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00909847 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00926697 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00920144 s, 114 MB/s 1048576 bytes (1.0 MB) copied, 0.0105504 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00776583 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00953886 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00155747 s, 673 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103647 s, 101 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg324-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93119 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72797 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.24@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.24@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.1875 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00636137 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00876827 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123113 s, 85.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00974366 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162459 s, 64.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100926 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0058452 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00658879 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0722751 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0750576 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0842406 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0811016 s, 12.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.166062 s, 6.3 MB/s , 0.17257 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.265233 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.255595 s, 4.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00141953 s, 739 MB/s 1048576 bytes (1.0 MB) copied, 0.00148839 s, 705 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022511 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00556957 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00258536 s, 406 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00933819 s, 112 MB/s 1048576 bytes (1.0 MB) copied, 0.0075756 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00134465 s, 780 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0117288 s, 89.4 MB/s 1048576 bytes (1.0 MB) copied, 0.00145065 s, 723 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106267 s, 98.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142225 s, 73.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100678 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00260545 s, 402 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279195 s, 37.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00149508 s, 701 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg324-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.90689 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72618 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.24@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg324-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.185583 s, 90.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114016 s, 92.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101282 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185109 s, 56.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140413 s, 74.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269415 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0296527 s, 35.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245621 s, 42.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250321 s, 41.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0687526 s, 15.3 MB/s , 0.0671602 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0818172 s, 12.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0824733 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0825319 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.077786 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0929109 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.090483 s, 11.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0063487 s, 165 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00609246 s, 172 MB/s , 0.00844684 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00614807 s, 171 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00805251 s, 130 MB/s , 0.00872469 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00839643 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114114 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00586138 s, 179 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00890618 s, 118 MB/s , 0.00978941 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00888199 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102136 s, 103 MB/s , 0.00981311 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00922053 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010556 s, 99.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg324-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93117 s, 21.3 MB/s Write runtime is 5 s, speed is 20.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.87663 s, 21.5 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg324-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.336935 s, 49.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00700944 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101879 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124308 s, 84.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208749 s, 50.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0178211 s, 58.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118064 s, 88.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174064 s, 60.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178238 s, 58.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.173186 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.189648 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188956 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188281 s, 5.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.269901 s, 3.9 MB/s , 0.265388 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.321241 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.315548 s, 3.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00973775 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00879198 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00908529 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00955593 s, 110 MB/s 1048576 bytes (1.0 MB) copied, 0.010384 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0089345 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00852146 s, 123 MB/s , 0.0108454 s, 96.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00603156 s, 174 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00260334 s, 403 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101626 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103811 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105725 s, 99.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0119837 s, 87.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0106811 s, 98.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134341 s, 78.1 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg324-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77553 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72759 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.91345 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72846 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg324-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.644634 s, 26.0 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00707032 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121724 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194346 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123765 s, 84.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140948 s, 74.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157201 s, 66.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102286 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176569 s, 59.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.370315 s, 2.8 MB/s 1048576 bytes (1.0 MB) copied, 0.364843 s, 2.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.374538 s, 2.8 MB/s , 0.365352 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.380089 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.352556 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.38737 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.363807 s, 2.9 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00381694 s, 275 MB/s , 0.00948524 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00531915 s, 197 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00854502 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00705414 s, 149 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00798257 s, 131 MB/s 1048576 bytes (1.0 MB) copied, 0.00898863 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00600125 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00745486 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00910227 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00928267 s, 113 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00940377 s, 112 MB/s , 0.0094652 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108662 s, 96.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113839 s, 92.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114978 s, 91.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg324-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8296 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4295 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg324-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.192768 s, 87.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132458 s, 79.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163142 s, 64.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194729 s, 53.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188402 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021802 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106112 s, 98.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108152 s, 97.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156208 s, 67.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0688249 s, 15.2 MB/s , 0.0699915 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0643203 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0592065 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0744139 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0795313 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.071609 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0659947 s, 15.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00816365 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00837398 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104488 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00866155 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00930297 s, 113 MB/s 1048576 bytes (1.0 MB) copied, 0.0104276 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00939051 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00947283 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00981413 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0095611 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00878756 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00892116 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112521 s, 93.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011239 s, 93.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123398 s, 85.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106055 s, 98.9 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg324-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92303 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72723 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg324-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.33807 s, 49.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168308 s, 62.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202297 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283728 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133574 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281072 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280308 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256838 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.029098 s, 36.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.17029 s, 6.2 MB/s , 0.16802 s, 6.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.310808 s, 3.4 MB/s , 0.322371 s, 3.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.362738 s, 2.9 MB/s , 0.357297 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.361073 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.378225 s, 2.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00586506 s, 179 MB/s 1048576 bytes (1.0 MB) copied, 0.00584011 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00597195 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00612068 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104637 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00977025 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0078296 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00800411 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00887554 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00914435 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00828432 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00885348 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00969693 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00876091 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104353 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111972 s, 93.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg324-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.76697 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72505 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.76409 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73146 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (355s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 13:27:34 (1736360854) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (3s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 13:27:37 (1736360857) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg324-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.84022 s, 273 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg324-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0125 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 13:28:27 (1736360907) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.10056 s, 167 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00992067 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015104 s, 69.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236358 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279719 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343334 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275517 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188357 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0370919 s, 28.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283733 s, 37.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0199122 s, 52.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413973 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0404263 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333341 s, 31.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff88012a60b800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319774 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.032239 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209294 s, 50.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102596 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00822482 s, 127 MB/s 1048576 bytes (1.0 MB) copied, 0.0103651 s, 101 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010573 s, 99.2 MB/s 1048576 bytes (1.0 MB) copied, 0.00336721 s, 311 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119691 s, 87.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011448 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00981855 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00919716 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984686 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00923976 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109644 s, 95.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00451224 s, 232 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff88012a60b800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107112 s, 97.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00161879 s, 648 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00150985 s, 694 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg324-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94561 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93296 s, 21.3 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.157111 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141793 s, 74.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192463 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155244 s, 67.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248275 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203185 s, 51.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193507 s, 54.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0192966 s, 54.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116034 s, 90.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281571 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220983 s, 47.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0202884 s, 51.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400521 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0073806 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0422734 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342827 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0293415 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00989937 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114301 s, 91.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00983123 s, 107 MB/s 1048576 bytes (1.0 MB) copied, 0.0113157 s, 92.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142402 s, 73.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128076 s, 81.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127719 s, 82.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120154 s, 87.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137333 s, 76.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160802 s, 65.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015773 s, 66.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015879 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161574 s, 64.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161278 s, 65.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187203 s, 56.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200976 s, 52.2 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg324-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg324-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93754 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97299 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (64s) == sanityn test 77o: Changing rank should not panic ====== 13:29:31 (1736360971) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (4s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 13:29:35 (1736360975) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (32s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 13:30:08 (1736361008) mds.MDS.mdt.nrs_policies=tbf oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg324-client: oleg324-server: ssh exited with exit code 36 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg324-client: oleg324-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (9s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 13:30:16 (1736361016) oleg324-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device pdsh@oleg324-client: oleg324-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (3s) == sanityn test 79: xattr: intent error ================== 13:30:19 (1736361019) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505431 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-08 13:30:19.000000000 -0500 Modify: 2025-01-08 13:30:19.000000000 -0500 Change: 2025-01-08 13:30:19.000000000 -0500 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 13:30:31 (1736361031) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (0s) == sanityn test 80b: Accessing directory during migration ========================================================== 13:30:31 (1736361031) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 13:30:32 (1736361032) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 13:30:33 (1736361033) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 13:30:34 (1736361034) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (0s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 13:30:34 (1736361034) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6858 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6858 PASS 82 (2s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 13:30:36 (1736361036) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 13:30:37 (1736361037) fail_loc=0x8000060b PASS 84 (6s) == sanityn test 90: open/create and unlink striped directory ========================================================== 13:30:43 (1736361043) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 13:30:44 (1736361044) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 13:30:45 (1736361045) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 13:30:46 (1736361046) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2629 0xa45 0 0 2645 0xa55 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2630 0xa46 0 0 2646 0xa56 0 rc1=0 and rc2=0 PASS 93 (8s) == sanityn test 94: signal vs CP callback race =========== 13:30:54 (1736361054) osc.lustre-OST0000-osc-ffff88012a60b800.idle_timeout=0 osc.lustre-OST0000-osc-ffff8801369f4000.idle_timeout=0 osc.lustre-OST0001-osc-ffff88012a60b800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8801369f4000.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0458277 s, 179 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 2.99582 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 4649 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0106778 s, 384 kB/s osc.lustre-OST0000-osc-ffff88012a60b800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8801369f4000.idle_timeout=debug osc.lustre-OST0001-osc-ffff88012a60b800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8801369f4000.idle_timeout=debug PASS 94 (9s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 13:31:03 (1736361063) llite.lustre-ffff88012a60b800.fast_read=0 llite.lustre-ffff8801369f4000.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00137926 s, 8.9 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff88012a60b800.fast_read=1 llite.lustre-ffff8801369f4000.fast_read=1 PASS 95b (8s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 13:31:11 (1736361071) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 13:31:12 (1736361072) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00162724 s, 2.5 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 13:31:14 (1736361074) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0182509 s, 115 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (1s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 13:31:15 (1736361075) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0282684 s, 74.2 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 13:31:17 (1736361077) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00134615 s, 8.9 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 13:31:19 (1736361079) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000735146 s, 5.6 MB/s PASS 101a (2s) == sanityn test 101b: Discard DoM data on rename ========= 13:31:21 (1736361081) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000564415 s, 7.3 MB/s PASS 101b (1s) == sanityn test 101c: Discard DoM data on close-unlink === 13:31:22 (1736361082) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000465879 s, 8.8 MB/s PASS 101c (3s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 13:31:25 (1736361085) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1736361086 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1736361088 Finishing test test23 at 1736361088 PASS 103 (6s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 13:31:31 (1736361091) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 13:31:32 (1736361092) fail_loc=0x416 PASS 105 (47s) == sanityn test 106a: Verify the btime via statx() ======= 13:32:20 (1736361140) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 13:32:20 (1736361140) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) == sanityn test 106c: Verify statx attributes mask ======= 13:32:21 (1736361141) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) == sanityn test 107a: Basic grouplock conflict =========== 13:32:22 (1736361142) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0570156 s, 184 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2635 0xa4b 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6858 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6858 First grouplock blocks second one PASS 107a (3s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 13:32:25 (1736361145) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0608414 s, 172 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2652 0xa5c 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6858 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) == sanityn test 108a: lseek: parallel updates ============ 13:32:33 (1736361153) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00119406 s, 8.4 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.222 s, 2.0 MB/s PASS 108a (6s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 13:32:39 (1736361159) 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions oleg324-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 11 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions oleg324-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 22 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions oleg324-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 23 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions oleg324-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 35 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions oleg324-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 47 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 192.168.203.124@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.124@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg324-client.virtnet /mnt/lustre (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg324-server' oleg324-server: oleg324-server.virtnet: executing load_modules_local oleg324-server: Loading modules from /home/green/git/lustre-release/lustre oleg324-server: detected 4 online CPUs by sysfs oleg324-server: Force libcfs to create 2 CPU partitions Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre Starting client: oleg324-client.virtnet: -o user_xattr,flock oleg324-server@tcp:/lustre /mnt/lustre2 PASS 109 (426s) == sanityn test 110: do not grant another lock on resend ========================================================== 13:39:45 (1736361585) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-08 13:39:46.000000000 -0500 Modify: 2025-01-08 13:39:46.000000000 -0500 Change: 2025-01-08 13:39:46.000000000 -0500 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg324-client.virtnet oleg324-client.virtnet PASS 110 (89s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 13:41:14 (1736361674) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 13:41:15 (1736361675) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 13:41:16 (1736361676) lustre-OST0000-osc-ffff8800a97cb800 active. lustre-OST0001-osc-ffff8800a97cb800 active. lustre-OST0000-osc-ffff8800a97cb800 active. lustre-OST0001-osc-ffff8800a97cb800 active. MGC192.168.203.124@tcp active. PASS 113 (3s) cleanup: ====================================================== == sanityn test complete, duration 3944 sec ============== 13:41:19 (1736361679) Stopping clients: oleg324-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg324-client.virtnet /mnt/lustre2 opts: