-----============= acceptance-small: sanityn ============----- Mon Jul 29 03:25:02 EDT 2024 excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Started clients oleg307-client.virtnet: 192.168.203.107@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt) oleg307-client.virtnet: executing check_config_client /mnt/lustre oleg307-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg307-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800aa4c3800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b3e32800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aa4c3800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b3e32800.idle_timeout=debug disable quota as required oleg307-server: oleg307-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115798 s, 90.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6837] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 03:25:11 (1722237911) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (2s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 03:25:13 (1722237913) -rw-r--r-- 1 root root 0 Jul 29 03:25 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (1s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 03:25:14 (1722237914) -rw-r--r-- 1 root root 0 Jul 29 03:25 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (2s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 03:25:16 (1722237916) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (1s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 03:25:17 (1722237917) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (2s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 03:25:19 (1722237919) total 1 -rwxrwxrwx 1 root root 0 Jul 29 03:25 f2a -rwxrwxrwx 1 root root 0 Jul 29 03:25 f2b total 1 -rwxrwxrwx 1 root root 0 Jul 29 03:25 f2a -rwxrwxrwx 1 root root 0 Jul 29 03:25 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00401941 s, 127 kB/s PASS 2e (2s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 03:25:21 (1722237921) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 03:25:22 (1722237922) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.171765 s, 12.2 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (2s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 03:25:24 (1722237924) PASS 3 (2s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 03:25:26 (1722237926) PASS 4 (2s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 03:25:29 (1722237929) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (1s) == sanityn test 6: remove of open file on other node ============================================================================ 03:25:30 (1722237930) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (2s) == sanityn test 7: remove of open directory on other node ======================================================================= 03:25:32 (1722237932) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (2s) == sanityn test 8: remove of open special file on other node ==================================================================== 03:25:34 (1722237934) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (1s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 03:25:35 (1722237935) PASS 9a (2s) == sanityn test 9b: append to striped sparse file ======== 03:25:37 (1722237937) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116177 s, 90.3 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0192225 s, 0.2 kB/s Data read (expecting 'foo'): foo PASS 9b (2s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 03:25:39 (1722237939) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00214094 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00183473 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00191959 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00200464 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00210804 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00184718 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00226717 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00214757 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0027161 s, 0.4 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00201575 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0019183 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00247875 s, 0.4 kB/s PASS 10a (2s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 03:25:41 (1722237941) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00327903 s, 937 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00895835 s, 457 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00022604 s, 13.6 MB/s PASS 10b (1s) == sanityn test 11: execution of file opened for write should return error ============================================================== 03:25:42 (1722237942) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6837 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (2s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 03:25:44 (1722237944) warning: '-runtime' deprecated, use '-t runtime' instead - link 2082 (time 1722237955.21 total 10.00 last 208.14) - link 3952 (time 1722237965.21 total 20.01 last 186.86) - link 5532 (time 1722237975.21 total 30.01 last 157.98) - link 6961 (time 1722237985.22 total 40.02 last 142.82) - link 9039 (time 1722237995.22 total 50.02 last 207.72) - link 10000 (time 1722237999.84 total 54.64 last 208.09) - link 12241 (time 1722238009.85 total 64.64 last 224.02) - link 14249 (time 1722238019.85 total 74.65 last 200.75) - link 16220 (time 1722238029.85 total 84.65 last 197.04) - link 18289 (time 1722238039.85 total 94.65 last 206.84) total: 19651 link in 100.00 seconds: 196.51 ops/second using seed 3282204037 running for 100 seconds - stat 10000 (time 1722237949 ; total 3 ; last 3) - stat 20000 (time 1722237952 ; total 6 ; last 3) - stat 30000 (time 1722237955 ; total 9 ; last 3) - stat 40000 (time 1722237958 ; total 12 ; last 3) - stat 50000 (time 1722237962 ; total 16 ; last 4) - stat 60000 (time 1722237965 ; total 19 ; last 3) - stat 70000 (time 1722237968 ; total 22 ; last 3) - stat 80000 (time 1722237971 ; total 25 ; last 3) - stat 90000 (time 1722237975 ; total 29 ; last 4) - stat 100000 (time 1722237978 ; total 32 ; last 3) - stat 110000 (time 1722237981 ; total 35 ; last 3) - stat 120000 (time 1722237985 ; total 39 ; last 4) - stat 130000 (time 1722237988 ; total 42 ; last 3) - stat 140000 (time 1722237991 ; total 45 ; last 3) - stat 150000 (time 1722237994 ; total 48 ; last 3) - stat 160000 (time 1722238000 ; total 54 ; last 6) - stat 170000 (time 1722238007 ; total 61 ; last 7) - stat 180000 (time 1722238015 ; total 69 ; last 8) - stat 190000 (time 1722238023 ; total 77 ; last 8) - stat 200000 (time 1722238032 ; total 86 ; last 9) - stat 210000 (time 1722238040 ; total 94 ; last 8) total: 218553 stats in 100 seconds: 2185.530029 stats/second - unlinked 0 (time 1722237996 ; total 0 ; last 0) - unlinked 10000 (time 1722238053 ; total 57 ; last 57) unlink(/mnt/lustre2/lockdir/lockfile19651) error: No such file or directory total: 19650 unlinks in 80 seconds: 245.625000 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (20434) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (20437) - No such process PASS 12 (133s) == sanityn test 13: test directory page revocation ======= 03:27:57 (1722238077) f13 f13 PASS 13 (1s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 03:27:58 (1722238078) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6837 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4253: 21805 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 03:28:00 (1722238080) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4253: 22432 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (1s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 03:28:01 (1722238081) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 23058 Terminated $DIR1/$tdir/sleep 60 PASS 14b (2s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 03:28:03 (1722238083) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 23687 Terminated $DIR1/$tdir/sleep 60 PASS 14c (1s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 03:28:04 (1722238084) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 24315 Terminated $DIR1/$tdir/sleep 60 PASS 14d (2s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 03:28:06 (1722238086) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7523328 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7523328kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (10s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 03:28:16 (1722238096) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.079828 s, 26.3 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0576744 s, 36.4 MB/s Chance of close/open is 1 in 50 Seed set to 8096 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000003[0] 1722238097.078370 mapwrite 0x207d77 thru 0x215d5c (0xdfe6 bytes) 000004[0] 1722238097.119902 mapread 0x19d432 thru 0x1a6dab (0x997a bytes) 000005[0] 1722238097.126615 mapwrite 0x7b574e thru 0x7bc7b6 (0x7069 bytes) 000006[0] 1722238097.152445 mapread 0x20a180 thru 0x20c382 (0x2203 bytes) truncating to largest ever: 0x3a9f4f 000008[1] 1722238097.153658 trunc from 0x7bc7b7 to 0x3a9f4e (0x412868 bytes) 000011[0] 1722238097.168331 mapread 0x3a525a thru 0x3a9f4e (0x4cf5 bytes) 000012[1] 1722238097.176479 mapread 0x04e61a thru 0x054dde (0x67c5 bytes) 000014[1] 1722238097.180819 write 0x5c9c72 thru 0x5d0f6b (0x72fa bytes) 000015[0] 1722238097.183297 write 0x830e3d thru 0x83b4ba (0xa67e bytes) 000016[1] 1722238097.188525 mapread 0x2dc8ea thru 0x2ddeea (0x1601 bytes) 000017[1] 1722238097.195538 read 0x5f413d thru 0x6037e8 (0xf6ac bytes) 000019[0] 1722238097.198057 trunc from 0x83b4bb to 0x1c8b52 (0x672968 bytes) 000022[0] 1722238097.211038 read 0x0d80c1 thru 0x0e5934 (0xd874 bytes) 000023[0] 1722238097.213201 write 0x3f239c thru 0x3f82c5 (0x5f2a bytes) 000026[0] 1722238097.219707 read 0x19420a thru 0x1a24fc (0xe2f3 bytes) 000028[1] 1722238097.231049 mapwrite 0x357778 thru 0x3648fc (0xd185 bytes) 000029[0] 1722238097.257314 trunc from 0x3f82c6 to 0x06816d (0x390158 bytes) 000031[0] 1722238097.263458 mapread 0x047adf thru 0x04e92a (0x6e4c bytes) 000032[1] 1722238097.274502 mapread 0x050077 thru 0x056ddb (0x6d65 bytes) 000035[0] 1722238097.289028 trunc from 0x06816e to 0x0d0b28 (0x689bb bytes) 000036[1] 1722238097.297150 mapwrite 0x0d6f51 thru 0x0e32bd (0xc36d bytes) truncating to largest ever: 0x5d59b8 000037[0] 1722238097.321908 trunc from 0x0e32be to 0x5d59b7 (0x4f26fa bytes) 000041[0] 1722238097.335652 mapread 0x18223a thru 0x184053 (0x1e1a bytes) truncating to largest ever: 0x5d7c75 000042[1] 1722238097.351823 trunc from 0x5d59b8 to 0x5d7c74 (0x22bd bytes) 000043[1] 1722238097.360419 write 0x55bc75 thru 0x56a3a7 (0xe733 bytes) 000044[0] 1722238097.363015 write 0x43a495 thru 0x449252 (0xedbe bytes) 000045[0] 1722238097.372542 mapwrite 0x73edca thru 0x749545 (0xa77c bytes) 000047[0] 1722238097.407790 write 0x4d4587 thru 0x4d4666 (0xe0 bytes) 000049[0] 1722238097.412338 mapwrite 0x0988e5 thru 0x0a43c5 (0xbae1 bytes) 000051[0] 1722238097.443334 mapread 0x3d5743 thru 0x3d8338 (0x2bf6 bytes) 000052[0] 1722238097.450365 read 0x1c12c1 thru 0x1d0612 (0xf352 bytes) 000056[1] 1722238097.480835 mapwrite 0x89f52a thru 0x8a76aa (0x8181 bytes) 000060[1] 1722238097.514617 mapread 0x6492be thru 0x657979 (0xe6bc bytes) 000061[1] 1722238097.520205 mapread 0x7e85f4 thru 0x7f4742 (0xc14f bytes) 000062[1] 1722238097.525730 write 0x3fc240 thru 0x3fec39 (0x29fa bytes) 000064[1] 1722238097.530869 read 0x5ea862 thru 0x5f203b (0x77da bytes) 000065[1] 1722238097.545761 trunc from 0x8a76ab to 0x49ed35 (0x408975 bytes) 000068[1] 1722238097.560426 mapwrite 0x4e678f thru 0x4e99fe (0x3270 bytes) 000069[1] 1722238097.585530 mapwrite 0x5c3e8a thru 0x5c8733 (0x48aa bytes) 000071[0] 1722238097.616969 read 0x589b1b thru 0x595846 (0xbd2c bytes) 000074[0] 1722238097.630297 mapwrite 0x917c59 thru 0x91c454 (0x47fc bytes) 000075[0] 1722238097.672698 read 0x046516 thru 0x055ec1 (0xf9ac bytes) 000077[0] 1722238097.688077 mapwrite 0x55f5d7 thru 0x56f480 (0xfeaa bytes) 000078[1] 1722238097.708592 read 0x86f36d thru 0x87e447 (0xf0db bytes) 000079[1] 1722238097.719237 mapwrite 0x6d676e thru 0x6d7883 (0x1116 bytes) 000081[1] 1722238097.743365 mapwrite 0x7447cd thru 0x74e59f (0x9dd3 bytes) 000084[0] 1722238097.782074 mapwrite 0x33ac69 thru 0x33b91b (0xcb3 bytes) 000085[0] 1722238097.802384 mapread 0x4e7a72 thru 0x4efb04 (0x8093 bytes) 000086[0] 1722238097.814417 trunc from 0x91c455 to 0x06b3c0 (0x8b1094 bytes) 000087[0] 1722238097.823014 mapwrite 0x5f6a07 thru 0x604a56 (0xe050 bytes) 000088[0] 1722238097.860299 mapread 0x128b41 thru 0x136c50 (0xe110 bytes) truncating to largest ever: 0x6c468e 000089[0] 1722238097.875772 trunc from 0x604a57 to 0x6c468d (0xbfc37 bytes) 000093[0] 1722238097.888376 mapwrite 0x07bc4c thru 0x07ebcd (0x2f82 bytes) 000096[0] 1722238097.906913 mapread 0x31cea9 thru 0x31fb7a (0x2cd2 bytes) 000099[0] 1722238097.911499 mapread 0x56a445 thru 0x56c5e4 (0x21a0 bytes) truncating to largest ever: 0x716c16 000103[0] 1722238097.915162 trunc from 0x6c468e to 0x716c15 (0x52588 bytes) 000104[0] 1722238097.919273 write 0x65429f thru 0x65cae9 (0x884b bytes) 000106[1] 1722238097.925981 mapwrite 0x51e582 thru 0x5285ab (0xa02a bytes) 000107[0] 1722238097.957060 mapwrite 0x43c2d2 thru 0x43ed01 (0x2a30 bytes) 000108[1] 1722238097.975552 trunc from 0x716c16 to 0x542d41 (0x1d3ed4 bytes) 000109[1] 1722238097.983424 trunc from 0x542d42 to 0x0bb260 (0x487ae1 bytes) 000111[0] 1722238097.986973 trunc from 0x0bb261 to 0x0a8433 (0x12e2d bytes) 000113[0] 1722238097.997598 trunc from 0x0a8434 to 0x58a09f (0x4e1c6c bytes) 000114[1] 1722238098.002196 mapread 0x4df81d thru 0x4eca27 (0xd20b bytes) 000115[0] 1722238098.019700 write 0x06c431 thru 0x0715b6 (0x5186 bytes) 000116[1] 1722238098.024055 mapread 0x56da50 thru 0x577337 (0x98e8 bytes) 000117[1] 1722238098.037836 write 0x7ea02f thru 0x7f69c9 (0xc99b bytes) 000118[0] 1722238098.044042 write 0x714c08 thru 0x721150 (0xc549 bytes) truncating to largest ever: 0x891175 000119[0] 1722238098.054035 trunc from 0x7f69ca to 0x891174 (0x9a7ab bytes) 000120[1] 1722238098.061389 trunc from 0x891175 to 0x2bccd2 (0x5d44a2 bytes) 000122[1] 1722238098.073074 trunc from 0x2bccd3 to 0x38e77d (0xd1aab bytes) 000125[0] 1722238098.083178 mapwrite 0x1cc8fa thru 0x1d7b4c (0xb253 bytes) 000126[0] 1722238098.116777 mapread 0x11678e thru 0x11eb82 (0x83f5 bytes) 000127[0] 1722238098.131726 mapread 0x0d1d35 thru 0x0d6240 (0x450c bytes) 000128[0] 1722238098.145717 trunc from 0x38e77e to 0x4040d8 (0x7595b bytes) 000129[0] 1722238098.151502 read 0x249412 thru 0x25226b (0x8e5a bytes) 000130[1] 1722238098.158757 mapwrite 0x2e98be thru 0x2f3b57 (0xa29a bytes) 000131[0] 1722238098.188080 mapwrite 0x9f123b thru 0x9f15fa (0x3c0 bytes) 000133[1] 1722238098.229162 read 0x2a612b thru 0x2a8e7f (0x2d55 bytes) 000134[1] 1722238098.239465 mapwrite 0x988a33 thru 0x98bbf3 (0x31c1 bytes) 000135[0] 1722238098.285986 trunc from 0x9f15fb to 0x06daa8 (0x983b52 bytes) 000136[1] 1722238098.300011 mapread 0x04c87e thru 0x058c53 (0xc3d6 bytes) 000137[0] 1722238098.320042 read 0x053045 thru 0x055190 (0x214c bytes) 000138[0] 1722238098.326706 mapread 0x04a96a thru 0x055fbe (0xb655 bytes) 000140[1] 1722238098.340532 mapwrite 0x21ed4e thru 0x21eef7 (0x1aa bytes) 000141[1] 1722238098.363735 trunc from 0x21eef8 to 0x84e698 (0x62f7a1 bytes) 000143[0] 1722238098.368785 read 0x130ffa thru 0x13c1ca (0xb1d1 bytes) 000145[0] 1722238098.382021 write 0x51126e thru 0x513d83 (0x2b16 bytes) 000146[0] 1722238098.389250 mapwrite 0x9cf9b9 thru 0x9df1b5 (0xf7fd bytes) 000147[0] 1722238098.435073 trunc from 0x9df1b6 to 0x500044 (0x4df171 bytes) 000148[0] 1722238098.441314 trunc from 0x500045 to 0x1cb706 (0x33493e bytes) 000149[0] 1722238098.446007 mapread 0x06d045 thru 0x070a8b (0x3a47 bytes) 000151[0] 1722238098.453498 read 0x074037 thru 0x07fe9e (0xbe68 bytes) 000152[0] 1722238098.459159 mapwrite 0x73fa16 thru 0x7461a4 (0x678f bytes) 000154[1] 1722238098.498720 mapread 0x339c68 thru 0x348d2e (0xf0c7 bytes) 000156[0] 1722238098.520103 write 0x3502a5 thru 0x35cb65 (0xc8c1 bytes) 000157[0] 1722238098.524789 trunc from 0x7461a5 to 0x30998e (0x43c816 bytes) 000158[1] 1722238098.529924 mapread 0x16678d thru 0x169ab8 (0x332c bytes) truncating to largest ever: 0x9b4b33 000159[1] 1722238098.540083 trunc from 0x30998f to 0x9b4b32 (0x6ab1a4 bytes) 000161[1] 1722238098.554592 read 0x916fa8 thru 0x91b8f1 (0x494a bytes) 000162[1] 1722238098.559871 write 0x425f0d thru 0x42bb7e (0x5c72 bytes) 000164[0] 1722238098.562416 mapwrite 0x126f26 thru 0x129ac0 (0x2b9b bytes) 000166[0] 1722238098.586405 trunc from 0x9b4b33 to 0x5fd381 (0x3b77b1 bytes) 000167[1] 1722238098.595209 mapwrite 0x7cb1ad thru 0x7d4296 (0x90ea bytes) 000169[0] 1722238098.629157 mapwrite 0x37d5b6 thru 0x38555e (0x7fa9 bytes) 000175[0] 1722238098.671243 read 0x39cc66 thru 0x3a2b82 (0x5f1d bytes) 000177[0] 1722238098.681739 mapwrite 0x4f0494 thru 0x4fad08 (0xa875 bytes) 000180[0] 1722238098.717408 mapread 0x4e6915 thru 0x4f3fb2 (0xd69e bytes) 000182[1] 1722238098.718153 mapread 0x468a57 thru 0x46cdaf (0x4359 bytes) 000183[1] 1722238098.731732 trunc from 0x7d4297 to 0x89c758 (0xc84c2 bytes) 000185[1] 1722238098.747044 mapwrite 0x181d35 thru 0x189eb4 (0x8180 bytes) 000186[1] 1722238098.773483 trunc from 0x89c759 to 0x8a6d9c (0xa644 bytes) 000190[1] 1722238098.780804 mapwrite 0x4ca667 thru 0x4cefdc (0x4976 bytes) 000192[1] 1722238098.802438 read 0x3cb685 thru 0x3ccaf1 (0x146d bytes) 000193[0] 1722238098.820934 trunc from 0x8a6d9d to 0x571111 (0x335c8b bytes) 000196[1] 1722238098.835843 trunc from 0x571112 to 0x24c155 (0x324fbc bytes) 000198[0] 1722238098.846119 read 0x0f1ec0 thru 0x0f30a3 (0x11e4 bytes) 000199[1] 1722238098.850328 mapread 0x12618d thru 0x134861 (0xe6d5 bytes) 000201[0] 1722238098.866410 read 0x02e74c thru 0x030fb6 (0x286b bytes) 000205[1] 1722238098.879540 write 0x6f4590 thru 0x6f468a (0xfb bytes) 000206[0] 1722238098.883149 mapwrite 0x6d88c6 thru 0x6dc18c (0x38c7 bytes) 000207[0] 1722238098.907681 write 0x4d35c4 thru 0x4d3653 (0x90 bytes) 000209[0] 1722238098.914285 mapread 0x2c2584 thru 0x2cc04a (0x9ac7 bytes) 000210[0] 1722238098.929564 read 0x401703 thru 0x405b84 (0x4482 bytes) 000212[1] 1722238098.941399 read 0x697798 thru 0x69c837 (0x50a0 bytes) 000213[0] 1722238098.953811 trunc from 0x6f468b to 0x6ce0a3 (0x265e7 bytes) 000217[1] 1722238098.970391 mapwrite 0x00a701 thru 0x010f4c (0x684c bytes) 000218[0] 1722238098.993974 read 0x3e3902 thru 0x3ec0ed (0x87ec bytes) 000219[1] 1722238098.999153 mapread 0x0046c4 thru 0x00d587 (0x8ec4 bytes) 000221[1] 1722238099.010877 trunc from 0x6ce0a4 to 0x699fab (0x340f8 bytes) 000222[1] 1722238099.018879 mapwrite 0x32f9fe thru 0x334056 (0x4659 bytes) 000223[0] 1722238099.038414 write 0x1e2510 thru 0x1ef6e1 (0xd1d2 bytes) 000225[0] 1722238099.048532 trunc from 0x699fac to 0x95f5b6 (0x2c560b bytes) 000226[0] 1722238099.055020 write 0x9be503 thru 0x9c2a9f (0x459d bytes) 000228[1] 1722238099.059395 write 0x406fe7 thru 0x414686 (0xd6a0 bytes) 000230[0] 1722238099.069249 trunc from 0x9c2aa0 to 0x6e3442 (0x2df65d bytes) 000231[1] 1722238099.076562 mapread 0x2f4306 thru 0x304259 (0xff54 bytes) 000233[0] 1722238099.101675 read 0x642a0c thru 0x65264c (0xfc41 bytes) 000234[1] 1722238099.106798 trunc from 0x6e3443 to 0x00f974 (0x6d3ace bytes) 000235[1] 1722238099.114478 trunc from 0x00f975 to 0x6abdbf (0x69c44b bytes) 000236[0] 1722238099.121038 trunc from 0x6abdc0 to 0x53544f (0x176970 bytes) 000239[0] 1722238099.131347 write 0x521fbb thru 0x526617 (0x465d bytes) 000240[1] 1722238099.136969 trunc from 0x535450 to 0x2bedbf (0x276690 bytes) 000241[1] 1722238099.148907 write 0x16d33e thru 0x1780d2 (0xad95 bytes) 000243[1] 1722238099.155627 write 0x6b454e thru 0x6be379 (0x9e2c bytes) 000245[0] 1722238099.157035 mapwrite 0x44e861 thru 0x455ef6 (0x7696 bytes) 000246[1] 1722238099.186295 mapwrite 0x64b07d thru 0x64f8c2 (0x4846 bytes) 000247[1] 1722238099.206869 read 0x10fba7 thru 0x11dc96 (0xe0f0 bytes) 000248[0] 1722238099.223040 mapread 0x197dcc thru 0x1a0812 (0x8a47 bytes) 000250[1] 1722238099.243312 mapwrite 0x855ea1 thru 0x86310a (0xd26a bytes) 000251[1] 1722238099.270471 write 0x6d7dd0 thru 0x6d8dac (0xfdd bytes) 000252[0] 1722238099.273802 write 0x9b3f40 thru 0x9b82a1 (0x4362 bytes) 000253[1] 1722238099.280675 mapread 0x2d1ae8 thru 0x2db455 (0x996e bytes) 000254[1] 1722238099.284950 write 0x118aa4 thru 0x11a05d (0x15ba bytes) 000256[1] 1722238099.292001 trunc from 0x9b82a2 to 0x4b35ae (0x504cf3 bytes) 000258[1] 1722238099.302033 write 0x7ab6a7 thru 0x7ae369 (0x2cc3 bytes) 000260[0] 1722238099.308202 mapread 0x2f0299 thru 0x2fc47b (0xc1e3 bytes) 000261[0] 1722238099.325962 trunc from 0x7ae36a to 0x95a5cb (0x1ac262 bytes) 000262[1] 1722238099.333725 write 0x77ecac thru 0x78a037 (0xb38c bytes) 000264[0] 1722238099.343174 trunc from 0x95a5cc to 0x7e5941 (0x174c8a bytes) 000265[0] 1722238099.351821 mapwrite 0x329fc9 thru 0x32ecbe (0x4cf6 bytes) 000266[0] 1722238099.375676 read 0x380482 thru 0x38fb9d (0xf71c bytes) 000267[0] 1722238099.389591 read 0x400a8a thru 0x403e72 (0x33e9 bytes) 000271[0] 1722238099.396163 mapwrite 0x5c25ac thru 0x5cb088 (0x8add bytes) 000272[0] 1722238099.410209 mapwrite 0x59150f thru 0x596033 (0x4b25 bytes) 000275[0] 1722238099.425680 read 0x5abcf9 thru 0x5b3a1e (0x7d26 bytes) 000277[0] 1722238099.428767 read 0x2b39d1 thru 0x2c295f (0xef8f bytes) 000278[1] 1722238099.433920 read 0x0b3dd5 thru 0x0bc4c4 (0x86f0 bytes) 000279[0] 1722238099.442302 mapwrite 0x089e88 thru 0x096d62 (0xcedb bytes) 000280[1] 1722238099.469699 trunc from 0x7e5942 to 0x4f3b75 (0x2f1dcc bytes) 000283[0] 1722238099.482814 write 0x1e5eb6 thru 0x1e8572 (0x26bd bytes) 000285[0] 1722238099.491289 trunc from 0x4f3b76 to 0x454662 (0x9f513 bytes) 000286[0] 1722238099.495987 read 0x43fe6d thru 0x4412d5 (0x1469 bytes) 000289[1] 1722238099.501622 mapwrite 0x977f94 thru 0x97df8b (0x5ff8 bytes) 000291[0] 1722238099.552680 write 0x1edf48 thru 0x1f2489 (0x4542 bytes) 000292[1] 1722238099.561851 mapread 0x366f2b thru 0x367996 (0xa6c bytes) 000295[1] 1722238099.573213 read 0x3d838c thru 0x3e2ca9 (0xa91e bytes) 000296[1] 1722238099.586366 read 0x518e0f thru 0x51f637 (0x6829 bytes) 000298[0] 1722238099.592542 trunc from 0x97df8c to 0x64fc69 (0x32e322 bytes) 000299[0] 1722238099.606921 mapread 0x4164be thru 0x4242c6 (0xde09 bytes) 000305[0] 1722238099.630970 mapwrite 0x268e96 thru 0x2748a7 (0xba12 bytes) 000306[1] 1722238099.655772 write 0x847410 thru 0x8528e7 (0xb4d8 bytes) 000307[0] 1722238099.661865 mapread 0x1014b1 thru 0x101aa0 (0x5f0 bytes) 000308[1] 1722238099.666897 read 0x21faaf thru 0x22c598 (0xcaea bytes) 000311[1] 1722238099.679930 trunc from 0x8528e8 to 0x90a035 (0xb774e bytes) 000314[1] 1722238099.691882 read 0x309d00 thru 0x30f455 (0x5756 bytes) 000315[0] 1722238099.714502 read 0x212bec thru 0x221315 (0xe72a bytes) 000317[0] 1722238099.734224 read 0x0795ef thru 0x07cd2b (0x373d bytes) 000318[0] 1722238099.736211 mapwrite 0x25e86f thru 0x26a1c9 (0xb95b bytes) 000322[0] 1722238099.778901 mapwrite 0x221a47 thru 0x2231d7 (0x1791 bytes) 000324[1] 1722238099.798675 mapwrite 0x929376 thru 0x92e16d (0x4df8 bytes) 000326[0] 1722238099.817815 write 0x26c01f thru 0x26f9b0 (0x3992 bytes) 000327[0] 1722238099.824221 write 0x142c8d thru 0x151d9c (0xf110 bytes) 000328[0] 1722238099.831516 mapwrite 0x6fcbbe thru 0x703e93 (0x72d6 bytes) 000332[1] 1722238099.854463 mapread 0x79f869 thru 0x7a3e0b (0x45a3 bytes) 000336[0] 1722238099.871746 write 0x4e7b98 thru 0x4e9b86 (0x1fef bytes) 000337[1] 1722238099.873966 mapwrite 0x80a014 thru 0x80c396 (0x2383 bytes) 000338[1] 1722238099.895094 write 0x979493 thru 0x981326 (0x7e94 bytes) 000339[1] 1722238099.900830 trunc from 0x981327 to 0x0f369a (0x88dc8c bytes) 000341[0] 1722238099.909071 trunc from 0x0f369b to 0x73b542 (0x647ea8 bytes) 000343[0] 1722238099.921103 write 0x85cb51 thru 0x86004f (0x34ff bytes) 000344[0] 1722238099.924588 trunc from 0x860050 to 0x489d98 (0x3d62b7 bytes) 000345[0] 1722238099.931742 trunc from 0x489d99 to 0x1f468b (0x29570d bytes) 000349[0] 1722238099.939415 mapwrite 0x6166f3 thru 0x618624 (0x1f32 bytes) 000350[1] 1722238099.961490 mapwrite 0x1f470d thru 0x203ed9 (0xf7cd bytes) 000352[0] 1722238100.002918 write 0x7be0b4 thru 0x7c81f6 (0xa143 bytes) 000354[0] 1722238100.010919 read 0x6741c0 thru 0x681bc2 (0xda03 bytes) 000355[1] 1722238100.019358 mapread 0x38bd95 thru 0x3925a7 (0x6813 bytes) 000357[1] 1722238100.033338 write 0x70b570 thru 0x70f0cd (0x3b5e bytes) 000358[0] 1722238100.040331 trunc from 0x7c81f7 to 0x4d3bf4 (0x2f4602 bytes) 000359[0] 1722238100.050040 read 0x11af7e thru 0x11e987 (0x3a0a bytes) 000361[0] 1722238100.068001 read 0x415845 thru 0x416f4a (0x1706 bytes) 000362[0] 1722238100.068278 mapwrite 0x9090a2 thru 0x9146c6 (0xb625 bytes) 000370[1] 1722238100.115613 mapwrite 0x4e10f8 thru 0x4e8227 (0x7130 bytes) 000371[1] 1722238100.142947 read 0x437c52 thru 0x43ba12 (0x3dc1 bytes) 000373[0] 1722238100.154142 trunc from 0x9146c7 to 0x90f6c3 (0x5003 bytes) 000374[1] 1722238100.160326 trunc from 0x90f6c4 to 0x2bd2bf (0x652404 bytes) 000376[0] 1722238100.171986 write 0x7d7050 thru 0x7da026 (0x2fd7 bytes) 000379[0] 1722238100.183103 trunc from 0x7da027 to 0x82b850 (0x5182a bytes) 000381[1] 1722238100.192747 write 0x42b385 thru 0x43a75a (0xf3d6 bytes) 000382[0] 1722238100.200067 mapread 0x17ea98 thru 0x18be25 (0xd38e bytes) 000383[0] 1722238100.217712 trunc from 0x82b851 to 0x5306a4 (0x2fb1ac bytes) 000384[1] 1722238100.229834 mapwrite 0x2c54f5 thru 0x2d0934 (0xb440 bytes) 000385[0] 1722238100.269223 trunc from 0x5306a5 to 0x12a6a8 (0x405ffc bytes) 000387[1] 1722238100.281236 read 0x07ea60 thru 0x08af22 (0xc4c3 bytes) 000390[0] 1722238100.295777 mapread 0x0919d2 thru 0x0a0997 (0xefc6 bytes) 000391[0] 1722238100.315354 mapread 0x0593e0 thru 0x05bcce (0x28ef bytes) 000392[0] 1722238100.322117 write 0x1196f8 thru 0x11cdec (0x36f5 bytes) 000394[1] 1722238100.324704 read 0x03854f thru 0x039ce1 (0x1793 bytes) 000395[1] 1722238100.327997 mapread 0x091dc2 thru 0x094d14 (0x2f53 bytes) 000396[1] 1722238100.329854 trunc from 0x12a6a9 to 0x31a579 (0x1efed1 bytes) 000398[1] 1722238100.345794 mapread 0x213a75 thru 0x21f33f (0xb8cb bytes) 000399[1] 1722238100.364654 mapwrite 0x784127 thru 0x785226 (0x1100 bytes) 000400[0] 1722238100.389328 trunc from 0x785227 to 0x371f5c (0x4132ca bytes) 000401[0] 1722238100.397687 mapread 0x15b50c thru 0x1699a4 (0xe499 bytes) 000403[0] 1722238100.415162 trunc from 0x371f5d to 0x8ae8ce (0x53c972 bytes) 000405[0] 1722238100.419867 read 0x2e47bc thru 0x2ef4ed (0xad32 bytes) 000409[1] 1722238100.446492 mapread 0x5b79b4 thru 0x5c4d44 (0xd391 bytes) 000410[0] 1722238100.467968 trunc from 0x8ae8cf to 0x95d035 (0xae767 bytes) 000412[0] 1722238100.478416 write 0x815afa thru 0x81bac1 (0x5fc8 bytes) 000414[1] 1722238100.482928 mapwrite 0x98aca4 thru 0x9968a5 (0xbc02 bytes) 000415[0] 1722238100.530053 trunc from 0x9968a6 to 0x4e87be (0x4ae0e7 bytes) 000420[0] 1722238100.546517 mapwrite 0x77abae thru 0x7864d3 (0xb926 bytes) 000421[1] 1722238100.582139 write 0x5e0f02 thru 0x5ee713 (0xd812 bytes) 000423[0] 1722238100.594289 mapread 0x7505ce thru 0x75f636 (0xf069 bytes) 000424[0] 1722238100.616199 read 0x21e033 thru 0x22bcff (0xdccd bytes) 000425[0] 1722238100.639770 read 0x780444 thru 0x78086b (0x428 bytes) 000426[0] 1722238100.640173 read 0x475ba8 thru 0x47a033 (0x448c bytes) 000427[1] 1722238100.644128 write 0x47fe69 thru 0x48e344 (0xe4dc bytes) 000431[1] 1722238100.662042 mapwrite 0x4ef42e thru 0x4f3c7e (0x4851 bytes) 000434[1] 1722238100.688652 read 0x499aa3 thru 0x49ec7b (0x51d9 bytes) 000436[1] 1722238100.701452 mapread 0x5b820e thru 0x5c81ca (0xffbd bytes) 000437[0] 1722238100.704558 mapread 0x322d4a thru 0x329c79 (0x6f30 bytes) 000438[1] 1722238100.706290 read 0x2fe36f thru 0x306efb (0x8b8d bytes) 000439[0] 1722238100.718811 mapwrite 0x89fa9a thru 0x8ade39 (0xe3a0 bytes) 000440[0] 1722238100.767519 trunc from 0x8ade3a to 0x5fc53f (0x2b18fa bytes) 000441[1] 1722238100.775706 mapwrite 0x859597 thru 0x8649d0 (0xb43a bytes) 000444[1] 1722238100.815385 read 0x60cf2b thru 0x61320a (0x62e0 bytes) 000445[0] 1722238100.823343 mapread 0x3415c8 thru 0x3508e4 (0xf31d bytes) 000447[1] 1722238100.847511 write 0x2394b5 thru 0x23c5f0 (0x313c bytes) 000448[0] 1722238100.850621 write 0x874187 thru 0x874c38 (0xab2 bytes) 000453[0] 1722238100.864236 trunc from 0x874c39 to 0x2b63f3 (0x5be845 bytes) 000457[1] 1722238100.877838 trunc from 0x2b63f4 to 0x00367a (0x2b2d79 bytes) 000458[0] 1722238100.885790 mapwrite 0x5bedcf thru 0x5c1ca1 (0x2ed3 bytes) 000459[1] 1722238100.924800 mapwrite 0x017158 thru 0x01cf5a (0x5e03 bytes) 000460[1] 1722238100.954901 write 0x32fce6 thru 0x3347d7 (0x4af2 bytes) 000461[0] 1722238100.961412 write 0x6354e3 thru 0x642145 (0xcc63 bytes) 000462[1] 1722238100.966596 mapread 0x350097 thru 0x359542 (0x94ac bytes) 000464[1] 1722238100.983934 write 0x82f1f3 thru 0x83820d (0x901b bytes) 000467[0] 1722238100.999095 mapread 0x100fb2 thru 0x103354 (0x23a3 bytes) 000468[0] 1722238101.010681 mapread 0x362988 thru 0x367f59 (0x55d2 bytes) 000471[0] 1722238101.025383 mapwrite 0x930933 thru 0x9318b1 (0xf7f bytes) 000472[1] 1722238101.056339 mapwrite 0x5772f2 thru 0x5858af (0xe5be bytes) 000473[0] 1722238101.094646 read 0x6a25a8 thru 0x6ae7f0 (0xc249 bytes) 000478[1] 1722238101.116454 mapwrite 0x6ad047 thru 0x6b3c93 (0x6c4d bytes) 000479[0] 1722238101.147454 trunc from 0x9318b2 to 0x694377 (0x29d53a bytes) 000480[0] 1722238101.159554 read 0x644e98 thru 0x645200 (0x369 bytes) 000481[1] 1722238101.165775 mapread 0x4c711a thru 0x4cca6e (0x5955 bytes) 000482[1] 1722238101.179031 mapwrite 0x0bf9cd thru 0x0cae76 (0xb4aa bytes) 000484[1] 1722238101.220619 read 0x4a6f56 thru 0x4b2a99 (0xbb44 bytes) 000486[1] 1722238101.230697 write 0x23cbc7 thru 0x243c59 (0x7093 bytes) 000491[0] 1722238101.249547 mapwrite 0x10cb07 thru 0x10f498 (0x2992 bytes) 000492[1] 1722238101.277484 write 0x49bd67 thru 0x49e277 (0x2511 bytes) 000493[0] 1722238101.284510 write 0x90e4c9 thru 0x910958 (0x2490 bytes) 000495[0] 1722238101.293720 mapread 0x69e709 thru 0x6a3329 (0x4c21 bytes) 000498[1] 1722238101.315802 read 0x8e3a65 thru 0x8f0224 (0xc7c0 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 8101 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[0] 1722238101.541790 mapwrite 0x045000 thru 0x047f5c (0x2f5d bytes) truncating to largest ever: 0x97dee9 000003[1] 1722238101.582216 trunc from 0x047f5d to 0x97dee8 (0x935f8c bytes) 000004[0] 1722238101.590549 trunc from 0x97dee9 to 0x855cb0 (0x128238 bytes) 000006[0] 1722238101.603182 read_OD 0x168000 thru 0x175fff (0xe000 bytes) 000009[1] 1722238101.610632 read 0x7b3000 thru 0x7b8cdf (0x5ce0 bytes) 000010[0] 1722238101.614875 write_OD 0x97d000 thru 0x984fff (0x8000 bytes) 000012[0] 1722238101.635250 trunc from 0x985000 to 0x8827b9 (0x102846 bytes) 000013[0] 1722238101.647619 read_OD 0x03e000 thru 0x047fff (0xa000 bytes) 000014[0] 1722238101.652043 read_OD 0x0ff000 thru 0x10dfff (0xf000 bytes) 000017[0] 1722238101.657117 write_OD 0x6dc000 thru 0x6defff (0x3000 bytes) 000018[0] 1722238101.681063 write_OD 0x088000 thru 0x08efff (0x7000 bytes) 000019[1] 1722238101.701389 read 0x445000 thru 0x44e8dc (0x98dd bytes) 000022[1] 1722238101.724768 mapread 0x2d0000 thru 0x2d190b (0x190c bytes) 000024[1] 1722238101.726729 trunc from 0x8827ba to 0x6de7d9 (0x1a3fe0 bytes) 000027[0] 1722238101.743343 trunc from 0x6de7da to 0x4c9424 (0x2153b5 bytes) 000029[1] 1722238101.749194 mapwrite 0x3f9000 thru 0x3fe9d8 (0x59d9 bytes) 000031[1] 1722238101.782737 mapwrite 0x07b000 thru 0x08597f (0xa980 bytes) 000032[0] 1722238101.817539 trunc from 0x4c9425 to 0x24653e (0x282ee6 bytes) 000034[1] 1722238101.828980 mapread 0x14a000 thru 0x157d97 (0xdd98 bytes) 000035[1] 1722238101.848765 mapwrite 0x76d000 thru 0x770fcd (0x3fce bytes) 000037[0] 1722238101.882049 write_OD 0x8e9000 thru 0x8effff (0x7000 bytes) 000040[0] 1722238101.903957 mapread 0x507000 thru 0x513c1a (0xcc1b bytes) 000041[1] 1722238101.937683 read 0x587000 thru 0x5944a6 (0xd4a7 bytes) 000042[0] 1722238101.957113 read_OD 0x1bf000 thru 0x1bffff (0x1000 bytes) 000044[1] 1722238101.958843 mapwrite 0x9f4000 thru 0x9feddc (0xaddd bytes) 000049[1] 1722238102.012127 mapread 0x227000 thru 0x22c596 (0x5597 bytes) 000051[0] 1722238102.019136 mapread 0x9b9000 thru 0x9c3c65 (0xac66 bytes) 000053[0] 1722238102.059561 write_OD 0x0fa000 thru 0x0fefff (0x5000 bytes) 000057[1] 1722238102.086017 mapwrite 0x7cf000 thru 0x7d2645 (0x3646 bytes) 000058[1] 1722238102.111360 mapread 0x1d0000 thru 0x1d7cdc (0x7cdd bytes) 000060[0] 1722238102.121051 trunc from 0x9feddd to 0x779d4a (0x285092 bytes) 000063[0] 1722238102.136725 read_OD 0x473000 thru 0x47dfff (0xb000 bytes) 000064[1] 1722238102.141218 write 0x729000 thru 0x734520 (0xb521 bytes) 000065[0] 1722238102.145279 read_OD 0x684000 thru 0x690fff (0xd000 bytes) truncating to largest ever: 0x9ce32a 000066[0] 1722238102.153178 trunc from 0x779d4b to 0x9ce329 (0x2545df bytes) 000067[1] 1722238102.162732 trunc from 0x9ce32a to 0x255820 (0x778b09 bytes) 000068[1] 1722238102.170353 trunc from 0x255821 to 0x64f79f (0x3f9f7f bytes) 000070[1] 1722238102.179418 mapwrite 0x5ac000 thru 0x5bb911 (0xf912 bytes) 000071[1] 1722238102.211111 mapwrite 0x684000 thru 0x687edf (0x3ee0 bytes) 000072[0] 1722238102.233689 read_OD 0x349000 thru 0x34efff (0x6000 bytes) 000073[0] 1722238102.240589 mapwrite 0x093000 thru 0x099dcc (0x6dcd bytes) 000074[1] 1722238102.280529 write 0x30d000 thru 0x319c65 (0xcc66 bytes) 000075[0] 1722238102.289740 mapwrite 0x94b000 thru 0x94c608 (0x1609 bytes) 000078[1] 1722238102.327328 mapwrite 0x58c000 thru 0x5916ae (0x56af bytes) 000084[1] 1722238102.358418 trunc from 0x94c609 to 0x4e7dab (0x46485d bytes) 000086[1] 1722238102.366434 read 0x287000 thru 0x28945e (0x245f bytes) 000087[0] 1722238102.377126 trunc from 0x4e7dac to 0x21b3aa (0x2cca01 bytes) 000088[1] 1722238102.386258 trunc from 0x21b3ab to 0x6961f9 (0x47ae4f bytes) 000091[0] 1722238102.400100 trunc from 0x6961fa to 0x335052 (0x3611a7 bytes) 000093[1] 1722238102.409359 trunc from 0x335053 to 0x85a67a (0x525628 bytes) 000095[1] 1722238102.420469 mapread 0x205000 thru 0x212a50 (0xda51 bytes) 000096[0] 1722238102.448565 write_OD 0x406000 thru 0x40dfff (0x8000 bytes) 000097[1] 1722238102.465279 mapread 0x247000 thru 0x2513cb (0xa3cc bytes) 000099[1] 1722238102.476227 write 0x55d000 thru 0x569c2c (0xcc2d bytes) 000101[0] 1722238102.482020 mapwrite 0x0cd000 thru 0x0d0046 (0x3047 bytes) 000102[0] 1722238102.505334 write_OD 0x957000 thru 0x95efff (0x8000 bytes) 000107[1] 1722238102.523967 write 0x832000 thru 0x8377ad (0x57ae bytes) 000108[0] 1722238102.537031 read_OD 0x807000 thru 0x807fff (0x1000 bytes) 000110[0] 1722238102.545086 trunc from 0x95f000 to 0x1930f3 (0x7cbf0c bytes) 000112[1] 1722238102.555155 mapwrite 0x1a8000 thru 0x1adbe8 (0x5be9 bytes) 000113[0] 1722238102.584354 mapread 0x0ab000 thru 0x0b55c5 (0xa5c6 bytes) 000114[0] 1722238102.600658 write_OD 0x4f6000 thru 0x4fafff (0x5000 bytes) 000115[1] 1722238102.616693 mapwrite 0x933000 thru 0x933e71 (0xe72 bytes) 000118[0] 1722238102.644922 read_OD 0x3f2000 thru 0x3f7fff (0x6000 bytes) 000119[0] 1722238102.649833 mapwrite 0x3a3000 thru 0x3a59e1 (0x29e2 bytes) 000123[1] 1722238102.687779 trunc from 0x933e72 to 0x457e4e (0x4dc023 bytes) 000124[0] 1722238102.697406 read_OD 0x05c000 thru 0x065fff (0xa000 bytes) 000130[0] 1722238102.703782 read_OD 0x374000 thru 0x37afff (0x7000 bytes) 000133[0] 1722238102.705623 mapread 0x44f000 thru 0x457db2 (0x8db3 bytes) 000134[1] 1722238102.712179 write 0x16e000 thru 0x178b56 (0xab57 bytes) 000136[0] 1722238102.718933 write_OD 0x530000 thru 0x53efff (0xf000 bytes) 000138[1] 1722238102.735386 trunc from 0x53f000 to 0x606e50 (0xc7e51 bytes) 000139[1] 1722238102.741366 read 0x48b000 thru 0x499211 (0xe212 bytes) 000142[0] 1722238102.751400 mapread 0x210000 thru 0x213e38 (0x3e39 bytes) 000144[1] 1722238102.763701 read 0x275000 thru 0x27ff20 (0xaf21 bytes) 000146[0] 1722238102.780162 mapread 0x109000 thru 0x118698 (0xf699 bytes) 000147[0] 1722238102.795703 read_OD 0x019000 thru 0x021fff (0x9000 bytes) 000148[0] 1722238102.797519 mapread 0x1d1000 thru 0x1d7b6d (0x6b6e bytes) 000150[1] 1722238102.821135 mapwrite 0x039000 thru 0x03e2e7 (0x52e8 bytes) 000151[1] 1722238102.853348 trunc from 0x606e51 to 0x0d0638 (0x536818 bytes) 000154[1] 1722238102.868988 write 0x292000 thru 0x2984ad (0x64ae bytes) 000157[1] 1722238102.877497 mapread 0x1d0000 thru 0x1d839b (0x839c bytes) 000158[0] 1722238102.889233 read_OD 0x0ae000 thru 0x0b2fff (0x5000 bytes) 000159[1] 1722238102.896618 mapread 0x218000 thru 0x21c171 (0x4172 bytes) 000161[1] 1722238102.902463 trunc from 0x2984ae to 0x89adbb (0x60290e bytes) 000165[1] 1722238102.915558 mapwrite 0x27a000 thru 0x28436b (0xa36c bytes) 000166[0] 1722238102.957305 write_OD 0x739000 thru 0x73cfff (0x4000 bytes) 000167[0] 1722238102.974178 trunc from 0x89adbc to 0x3ee022 (0x4acd99 bytes) 000169[0] 1722238102.980333 mapread 0x0ff000 thru 0x106d76 (0x7d77 bytes) 000171[1] 1722238102.997737 read 0x2bf000 thru 0x2c8ec2 (0x9ec3 bytes) 000172[0] 1722238103.011947 mapread 0x166000 thru 0x167fd9 (0x1fda bytes) 000173[1] 1722238103.014381 mapread 0x04d000 thru 0x05374c (0x674d bytes) 000174[1] 1722238103.022320 read 0x0b3000 thru 0x0b836b (0x536c bytes) 000176[1] 1722238103.034436 write 0x0db000 thru 0x0e5fa2 (0xafa3 bytes) 000179[0] 1722238103.046786 read_OD 0x09a000 thru 0x0a1fff (0x8000 bytes) 000181[1] 1722238103.053512 write 0x48f000 thru 0x49b2fc (0xc2fd bytes) 000182[1] 1722238103.059106 write 0x53b000 thru 0x541e1f (0x6e20 bytes) 000183[1] 1722238103.059766 mapwrite 0x845000 thru 0x84d020 (0x8021 bytes) 000185[0] 1722238103.099112 read_OD 0x7e0000 thru 0x7edfff (0xe000 bytes) 000188[0] 1722238103.112132 write_OD 0x119000 thru 0x123fff (0xb000 bytes) 000190[1] 1722238103.131446 mapwrite 0x7dd000 thru 0x7e46c5 (0x76c6 bytes) 000191[1] 1722238103.156613 read 0x51b000 thru 0x522a82 (0x7a83 bytes) 000194[1] 1722238103.174207 read 0x790000 thru 0x793005 (0x3006 bytes) 000199[1] 1722238103.180121 write 0x2a1000 thru 0x2a7594 (0x6595 bytes) 000201[0] 1722238103.181641 write_OD 0x081000 thru 0x086fff (0x6000 bytes) 000203[0] 1722238103.200364 mapwrite 0x311000 thru 0x31ec03 (0xdc04 bytes) 000205[0] 1722238103.250835 mapread 0x482000 thru 0x48a0db (0x80dc bytes) 000206[1] 1722238103.263080 trunc from 0x84d021 to 0x0f9107 (0x753f19 bytes) 000211[0] 1722238103.279653 mapwrite 0x374000 thru 0x37d167 (0x9168 bytes) 000213[1] 1722238103.314875 trunc from 0x37d168 to 0x80e9cd (0x491866 bytes) 000214[1] 1722238103.324136 trunc from 0x80e9ce to 0x374073 (0x49a95a bytes) 000217[0] 1722238103.332556 read_OD 0x037000 thru 0x03cfff (0x6000 bytes) 000219[0] 1722238103.338909 write_OD 0x31d000 thru 0x31ffff (0x3000 bytes) 000220[1] 1722238103.353312 read 0x052000 thru 0x05dd50 (0xbd51 bytes) 000222[0] 1722238103.366882 read_OD 0x215000 thru 0x222fff (0xe000 bytes) 000223[0] 1722238103.368950 write_OD 0x23a000 thru 0x23afff (0x1000 bytes) 000224[1] 1722238103.385129 mapread 0x2b2000 thru 0x2bc077 (0xa078 bytes) 000225[0] 1722238103.397689 trunc from 0x374074 to 0x917b8e (0x5a3b1b bytes) 000228[0] 1722238103.417318 trunc from 0x917b8f to 0x63a1ef (0x2dd99f bytes) 000229[1] 1722238103.421280 mapwrite 0x427000 thru 0x428eaa (0x1eab bytes) 000231[1] 1722238103.456633 write 0x4f0000 thru 0x4fec27 (0xec28 bytes) 000232[1] 1722238103.457189 read 0x2f6000 thru 0x3031be (0xd1bf bytes) 000233[0] 1722238103.462048 mapwrite 0x6a6000 thru 0x6acc7f (0x6c80 bytes) 000234[1] 1722238103.507174 mapread 0x11a000 thru 0x1242ab (0xa2ac bytes) 000235[0] 1722238103.529342 trunc from 0x6acc80 to 0x21da04 (0x48f27b bytes) 000236[1] 1722238103.536107 write 0x4f6000 thru 0x4fe3cf (0x83d0 bytes) 000238[0] 1722238103.545755 trunc from 0x4fe3d0 to 0x06bf6f (0x492460 bytes) 000239[1] 1722238103.556180 mapwrite 0x1a2000 thru 0x1ae553 (0xc554 bytes) 000244[0] 1722238103.603064 read_OD 0x0d7000 thru 0x0e4fff (0xe000 bytes) 000248[1] 1722238103.610310 write 0x5f4000 thru 0x5fd773 (0x9774 bytes) 000249[1] 1722238103.614213 mapread 0x4b4000 thru 0x4c2d9f (0xeda0 bytes) 000250[0] 1722238103.627619 mapread 0x394000 thru 0x394e86 (0xe87 bytes) 000252[1] 1722238103.652618 mapread 0x5ca000 thru 0x5d9e01 (0xfe02 bytes) 000253[1] 1722238103.670268 trunc from 0x5fd774 to 0x3c8494 (0x2352df bytes) 000255[1] 1722238103.681693 mapread 0x05c000 thru 0x064400 (0x8401 bytes) 000257[1] 1722238103.698541 mapread 0x26d000 thru 0x2793b6 (0xc3b7 bytes) 000258[0] 1722238103.710605 trunc from 0x3c8495 to 0x75a781 (0x3922ed bytes) 000260[1] 1722238103.717235 write 0x2b9000 thru 0x2c0cd9 (0x7cda bytes) 000261[0] 1722238103.727142 mapread 0x6e8000 thru 0x6ea706 (0x2707 bytes) 000262[0] 1722238103.744470 read_OD 0x1e5000 thru 0x1ebfff (0x7000 bytes) 000265[0] 1722238103.748303 read_OD 0x50e000 thru 0x50efff (0x1000 bytes) 000268[0] 1722238103.750029 write_OD 0x57e000 thru 0x585fff (0x8000 bytes) 000269[0] 1722238103.767159 mapread 0x0d0000 thru 0x0d1817 (0x1818 bytes) 000270[0] 1722238103.780889 mapread 0x578000 thru 0x57c891 (0x4892 bytes) 000271[0] 1722238103.781179 mapread 0x399000 thru 0x3a21d7 (0x91d8 bytes) 000273[1] 1722238103.791204 mapread 0x271000 thru 0x275ea8 (0x4ea9 bytes) 000278[1] 1722238103.796458 read 0x701000 thru 0x701bf4 (0xbf5 bytes) 000280[0] 1722238103.800272 mapwrite 0x4d3000 thru 0x4d8625 (0x5626 bytes) 000281[1] 1722238103.827673 write 0x561000 thru 0x569a58 (0x8a59 bytes) 000282[0] 1722238103.833667 mapread 0x4ae000 thru 0x4b7c5f (0x9c60 bytes) 000283[1] 1722238103.848920 trunc from 0x75a782 to 0x2c66b0 (0x4940d1 bytes) 000284[0] 1722238103.857614 write_OD 0x294000 thru 0x298fff (0x5000 bytes) 000288[0] 1722238103.872309 trunc from 0x2c66b1 to 0x1f496b (0xd1d45 bytes) 000289[0] 1722238103.877349 mapwrite 0x1f5000 thru 0x1f7cfe (0x2cff bytes) 000290[0] 1722238103.899092 mapread 0x10e000 thru 0x11442f (0x6430 bytes) 000291[1] 1722238103.909165 read 0x159000 thru 0x1607ee (0x77ef bytes) 000292[1] 1722238103.916443 read 0x048000 thru 0x055325 (0xd326 bytes) 000293[1] 1722238103.925916 read 0x11f000 thru 0x12829b (0x929c bytes) 000294[0] 1722238103.926144 trunc from 0x1f7cff to 0x878f6d (0x68126f bytes) 000296[0] 1722238103.934537 read_OD 0x1c4000 thru 0x1c7fff (0x4000 bytes) 000298[1] 1722238103.939490 read 0x2a7000 thru 0x2ad83f (0x6840 bytes) 000300[0] 1722238103.961859 read_OD 0x5d0000 thru 0x5d8fff (0x9000 bytes) 000304[0] 1722238103.964647 write_OD 0x477000 thru 0x47bfff (0x5000 bytes) 000305[0] 1722238103.986637 trunc from 0x878f6e to 0x0cca40 (0x7ac52d bytes) 000306[0] 1722238103.996872 trunc from 0x0cca41 to 0x51c309 (0x44f8c9 bytes) 000311[1] 1722238104.013463 read 0x2fd000 thru 0x302f4e (0x5f4f bytes) 000314[0] 1722238104.027303 mapread 0x1cd000 thru 0x1cfe85 (0x2e86 bytes) 000316[0] 1722238104.031398 write_OD 0x912000 thru 0x919fff (0x8000 bytes) 000317[0] 1722238104.053309 trunc from 0x91a000 to 0x0b5814 (0x8647eb bytes) 000321[1] 1722238104.067911 read 0x053000 thru 0x05802f (0x5030 bytes) 000323[0] 1722238104.073854 trunc from 0x0b5815 to 0x71da1f (0x66820b bytes) 000325[0] 1722238104.084819 read_OD 0x0cb000 thru 0x0d7fff (0xd000 bytes) 000327[0] 1722238104.088557 trunc from 0x71da20 to 0x5a9803 (0x17421c bytes) 000328[1] 1722238104.094194 trunc from 0x5a9804 to 0x15cedd (0x44c926 bytes) 000329[1] 1722238104.099974 write 0x64c000 thru 0x64fc14 (0x3c15 bytes) 000330[1] 1722238104.105605 trunc from 0x64fc15 to 0x6ad1ab (0x5d597 bytes) 000332[0] 1722238104.115110 read_OD 0x0ef000 thru 0x0fbfff (0xd000 bytes) 000333[0] 1722238104.122019 trunc from 0x6ad1ac to 0x15d401 (0x54fdaa bytes) 000336[0] 1722238104.132951 write_OD 0x38a000 thru 0x38afff (0x1000 bytes) 000337[0] 1722238104.146323 read_OD 0x1c3000 thru 0x1c8fff (0x6000 bytes) 000341[0] 1722238104.148981 trunc from 0x38b000 to 0x4aae2f (0x11fe30 bytes) 000344[1] 1722238104.157951 mapread 0x147000 thru 0x14db6b (0x6b6c bytes) 000352[1] 1722238104.179948 read 0x294000 thru 0x29497f (0x980 bytes) 000353[0] 1722238104.180160 trunc from 0x4aae30 to 0x927e33 (0x47d004 bytes) 000354[0] 1722238104.193169 trunc from 0x927e34 to 0x8532a7 (0xd4b8c bytes) 000355[0] 1722238104.197667 mapwrite 0x83f000 thru 0x8465df (0x75e0 bytes) 000356[1] 1722238104.242612 trunc from 0x8532a8 to 0x040e8a (0x81241d bytes) 000358[1] 1722238104.259518 read 0x02f000 thru 0x03d4aa (0xe4ab bytes) 000359[0] 1722238104.262478 mapread 0x038000 thru 0x03b1ac (0x31ad bytes) 000363[0] 1722238104.277711 mapread 0x015000 thru 0x022309 (0xd30a bytes) 000366[0] 1722238104.293864 write_OD 0x7fe000 thru 0x80bfff (0xe000 bytes) 000369[1] 1722238104.311237 read_OD 0x7e6000 thru 0x7ecfff (0x7000 bytes) 000370[0] 1722238104.313255 mapwrite 0x9a2000 thru 0x9ae06b (0xc06c bytes) 000372[1] 1722238104.352669 mapread 0x7b8000 thru 0x7bc702 (0x4703 bytes) 000374[1] 1722238104.360067 read_OD 0x89e000 thru 0x8abfff (0xe000 bytes) 000375[1] 1722238104.363480 read_OD 0x13a000 thru 0x13dfff (0x4000 bytes) 000376[0] 1722238104.364801 mapread 0x74f000 thru 0x75c928 (0xd929 bytes) 000377[1] 1722238104.379469 mapwrite 0x04a000 thru 0x059389 (0xf38a bytes) 000378[0] 1722238104.456117 trunc from 0x9ae06c to 0x731b18 (0x27c553 bytes) 000379[1] 1722238104.471857 trunc from 0x731b19 to 0x731592 (0x586 bytes) 000382[1] 1722238104.484862 mapread 0x5b7000 thru 0x5bee4a (0x7e4b bytes) 000383[0] 1722238104.496965 write_OD 0x5ad000 thru 0x5b4fff (0x8000 bytes) 000385[1] 1722238104.514674 trunc from 0x731593 to 0x0b4be7 (0x67c9ab bytes) 000387[1] 1722238104.521372 read_OD 0x02e000 thru 0x033fff (0x6000 bytes) 000388[1] 1722238104.525995 mapwrite 0x677000 thru 0x684a5c (0xda5d bytes) 000389[0] 1722238104.573277 mapwrite 0x6e9000 thru 0x6ef81f (0x6820 bytes) 000393[0] 1722238104.618813 mapwrite 0x7ba000 thru 0x7c6098 (0xc099 bytes) 000395[0] 1722238104.657251 write_OD 0x1e0000 thru 0x1e9fff (0xa000 bytes) 000396[0] 1722238104.672836 mapread 0x7c3000 thru 0x7c57b7 (0x27b8 bytes) 000399[0] 1722238104.677138 mapwrite 0x3a5000 thru 0x3a5c50 (0xc51 bytes) 000400[0] 1722238104.696534 trunc from 0x7c6099 to 0x914c5c (0x14ebc4 bytes) 000401[0] 1722238104.702258 read_OD 0x7be000 thru 0x7c3fff (0x6000 bytes) 000402[1] 1722238104.708203 mapread 0x830000 thru 0x8383b9 (0x83ba bytes) 000403[1] 1722238104.721510 write_OD 0x1ec000 thru 0x1f2fff (0x7000 bytes) 000407[0] 1722238104.739067 read_OD 0x49d000 thru 0x49ffff (0x3000 bytes) 000408[0] 1722238104.740461 mapread 0x41d000 thru 0x42c937 (0xf938 bytes) 000413[1] 1722238104.773175 write_OD 0x155000 thru 0x162fff (0xe000 bytes) 000414[0] 1722238104.789916 trunc from 0x914c5d to 0x946597 (0x3193b bytes) 000416[0] 1722238104.799251 mapread 0x02f000 thru 0x0362b1 (0x72b2 bytes) 000418[0] 1722238104.808778 write_OD 0x3e5000 thru 0x3e6fff (0x2000 bytes) 000420[0] 1722238104.824792 mapread 0x4bd000 thru 0x4cb28d (0xe28e bytes) 000421[0] 1722238104.843928 read_OD 0x544000 thru 0x54ffff (0xc000 bytes) 000426[0] 1722238104.846385 mapwrite 0x926000 thru 0x9341c2 (0xe1c3 bytes) 000427[0] 1722238104.882136 write_OD 0x980000 thru 0x986fff (0x7000 bytes) 000428[1] 1722238104.900527 mapread 0x5fc000 thru 0x5fd17a (0x117b bytes) 000429[1] 1722238104.903445 mapwrite 0x855000 thru 0x85e39f (0x93a0 bytes) 000430[1] 1722238104.931320 write_OD 0x0c0000 thru 0x0c8fff (0x9000 bytes) 000432[0] 1722238104.949121 mapread 0x610000 thru 0x6136a2 (0x36a3 bytes) 000436[0] 1722238104.954131 read_OD 0x3d7000 thru 0x3d9fff (0x3000 bytes) 000437[0] 1722238104.955738 write_OD 0x5b0000 thru 0x5bbfff (0xc000 bytes) 000439[1] 1722238104.976537 trunc from 0x987000 to 0x6493bf (0x33dc40 bytes) 000440[0] 1722238104.984354 trunc from 0x6493c0 to 0x7d8bb7 (0x18f7f8 bytes) 000441[0] 1722238104.991304 read_OD 0x63f000 thru 0x649fff (0xb000 bytes) 000443[1] 1722238104.997084 trunc from 0x7d8bb8 to 0x98b5e1 (0x1b2a2a bytes) 000444[0] 1722238105.008148 mapread 0x75b000 thru 0x7626a7 (0x76a8 bytes) 000445[0] 1722238105.029870 trunc from 0x98b5e2 to 0x694ba1 (0x2f6a40 bytes) 000448[1] 1722238105.047801 mapread 0x03a000 thru 0x043fcf (0x9fd0 bytes) 000452[0] 1722238105.065802 mapwrite 0x35d000 thru 0x3622e2 (0x52e3 bytes) 000453[1] 1722238105.090924 trunc from 0x694ba2 to 0x03eba6 (0x655ffb bytes) 000457[1] 1722238105.102744 mapread 0x00c000 thru 0x00d191 (0x1192 bytes) 000459[0] 1722238105.107766 trunc from 0x03eba7 to 0x4c4fc3 (0x48641d bytes) 000464[0] 1722238105.123305 read_OD 0x1c6000 thru 0x1c6fff (0x1000 bytes) 000466[0] 1722238105.127210 read_OD 0x092000 thru 0x097fff (0x6000 bytes) 000468[1] 1722238105.129538 read_OD 0x04c000 thru 0x058fff (0xd000 bytes) 000470[1] 1722238105.131673 read_OD 0x0f0000 thru 0x0f6fff (0x7000 bytes) 000472[0] 1722238105.133986 mapwrite 0x406000 thru 0x415fba (0xffbb bytes) 000475[0] 1722238105.176638 write_OD 0x67b000 thru 0x688fff (0xe000 bytes) 000476[0] 1722238105.195002 mapread 0x624000 thru 0x626262 (0x2263 bytes) 000477[1] 1722238105.198692 write_OD 0x248000 thru 0x248fff (0x1000 bytes) 000478[1] 1722238105.213284 mapwrite 0x6a6000 thru 0x6af8f6 (0x98f7 bytes) 000480[1] 1722238105.261943 write_OD 0x6e7000 thru 0x6e8fff (0x2000 bytes) 000482[1] 1722238105.280848 mapread 0x3ee000 thru 0x3f6438 (0x8439 bytes) 000483[1] 1722238105.291015 mapread 0x54b000 thru 0x557234 (0xc235 bytes) 000486[1] 1722238105.302821 read_OD 0x1ad000 thru 0x1b2fff (0x6000 bytes) 000487[0] 1722238105.304466 read_OD 0x540000 thru 0x54dfff (0xe000 bytes) 000488[1] 1722238105.306231 mapread 0x26d000 thru 0x27530c (0x830d bytes) 000489[1] 1722238105.326038 mapwrite 0x393000 thru 0x3a09b3 (0xd9b4 bytes) 000490[1] 1722238105.374068 write_OD 0x810000 thru 0x81afff (0xb000 bytes) 000491[0] 1722238105.389871 read_OD 0x26c000 thru 0x278fff (0xd000 bytes) 000492[0] 1722238105.392568 mapread 0x789000 thru 0x798c7a (0xfc7b bytes) 000493[0] 1722238105.413004 trunc from 0x81b000 to 0x1283bc (0x6f2c43 bytes) 000494[0] 1722238105.422674 mapread 0x103000 thru 0x10f273 (0xc274 bytes) 000498[0] 1722238105.441931 read_OD 0x05b000 thru 0x05efff (0x4000 bytes) 000499[0] 1722238105.444807 trunc from 0x1283bd to 0x1723f0 (0x4a034 bytes) All operations completed A-OK! PASS 16a (10s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 03:28:26 (1722238106) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0916703 s, 22.9 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0625607 s, 33.5 MB/s Chance of close/open is 1 in 50 Seed set to 8107 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x2a6677 000001[1] 1722238107.189299 trunc from 00000000 to 0x2a6676 (0x2a6677 bytes) 000002[1] 1722238107.195973 mapwrite 0x946134 thru 0x948096 (0x1f63 bytes) 000004[1] 1722238107.230644 write 0x6554b3 thru 0x655ad1 (0x61f bytes) 000008[0] 1722238107.241805 read 0x758f9d thru 0x7597fb (0x85f bytes) 000009[0] 1722238107.247703 read 0x110d68 thru 0x11191c (0xbb5 bytes) 000012[0] 1722238107.267815 mapread 0x7e54ff thru 0x7e5f3b (0xa3d bytes) 000016[0] 1722238107.273798 mapread 0x2b6bd4 thru 0x2b88c1 (0x1cee bytes) 000017[0] 1722238107.313651 read 0x4bbfda thru 0x4bc0a0 (0xc7 bytes) truncating to largest ever: 0x9a9f43 000018[0] 1722238107.313990 trunc from 0x948097 to 0x9a9f42 (0x61eac bytes) 000021[0] 1722238107.335108 mapread 0x0c4e9d thru 0x0c5599 (0x6fd bytes) 000022[1] 1722238107.337927 trunc from 0x9a9f43 to 0x5fc1e4 (0x3add5e bytes) 000024[0] 1722238107.352067 trunc from 0x5fc1e5 to 0x4934c1 (0x168d23 bytes) 000027[0] 1722238107.367903 trunc from 0x4934c2 to 0x4e91a1 (0x55ce0 bytes) truncating to largest ever: 0x9bdcb4 000028[0] 1722238107.375391 trunc from 0x4e91a2 to 0x9bdcb3 (0x4d4b12 bytes) 000029[1] 1722238107.380132 trunc from 0x9bdcb4 to 0x811667 (0x1ac64c bytes) 000030[1] 1722238107.386606 trunc from 0x811668 to 0x2ae0d5 (0x563592 bytes) 000031[1] 1722238107.389085 mapread 0x06dbd9 thru 0x06dcd0 (0xf8 bytes) 000032[0] 1722238107.394345 mapread 0x007faf thru 0x008d11 (0xd63 bytes) 000034[1] 1722238107.405098 mapwrite 0x151e25 thru 0x152ec9 (0x10a5 bytes) 000036[0] 1722238107.424597 write 0x4ce824 thru 0x4cf6ac (0xe89 bytes) 000038[0] 1722238107.434680 trunc from 0x4cf6ad to 0x6ebec5 (0x21c819 bytes) 000039[1] 1722238107.441526 mapread 0x2641b5 thru 0x264c7a (0xac6 bytes) 000041[0] 1722238107.452517 mapread 0x0ba8d1 thru 0x0bafb5 (0x6e5 bytes) 000042[0] 1722238107.456619 read 0x655514 thru 0x656df7 (0x18e4 bytes) 000043[0] 1722238107.466517 mapread 0x11c482 thru 0x11c61b (0x19a bytes) 000046[0] 1722238107.471243 read 0x4630b9 thru 0x4638cf (0x817 bytes) 000047[1] 1722238107.486626 read 0x4f7a87 thru 0x4f7b6b (0xe5 bytes) 000048[1] 1722238107.490881 write 0x96de7e thru 0x96e73d (0x8c0 bytes) 000050[1] 1722238107.500698 read 0x38fea6 thru 0x391382 (0x14dd bytes) 000051[1] 1722238107.517140 write 0x33a417 thru 0x33b71a (0x1304 bytes) 000052[1] 1722238107.520244 read 0x226b63 thru 0x228881 (0x1d1f bytes) 000053[0] 1722238107.531608 mapwrite 0x861784 thru 0x861a21 (0x29e bytes) 000054[0] 1722238107.560882 trunc from 0x96e73e to 0x4db603 (0x49313a bytes) 000055[0] 1722238107.572293 mapread 0x0fd5ef thru 0x0fe10d (0xb1f bytes) 000056[1] 1722238107.578867 trunc from 0x4db604 to 0x83b432 (0x35fe2f bytes) 000057[1] 1722238107.587294 write 0x07cf65 thru 0x07da6e (0xb0a bytes) 000060[1] 1722238107.599759 read 0x35496f thru 0x356104 (0x1796 bytes) 000061[0] 1722238107.619361 read 0x2ea7f5 thru 0x2eb3e2 (0xbee bytes) 000062[1] 1722238107.627916 read 0x2ee278 thru 0x2f022f (0x1fb8 bytes) 000064[0] 1722238107.649939 mapread 0x50b429 thru 0x50bb7a (0x752 bytes) 000070[1] 1722238107.676251 write 0x8fa2e8 thru 0x8fb374 (0x108d bytes) 000072[1] 1722238107.686193 read 0x1c2bae thru 0x1c3f2e (0x1381 bytes) 000073[1] 1722238107.707712 write 0x249df1 thru 0x24a4b7 (0x6c7 bytes) 000074[1] 1722238107.710596 mapread 0x3cd3b2 thru 0x3cdee9 (0xb38 bytes) 000075[1] 1722238107.713702 trunc from 0x8fb375 to 0x778aac (0x1828c8 bytes) 000084[1] 1722238107.736725 write 0x7bb66e thru 0x7bc679 (0x100c bytes) 000086[1] 1722238107.739775 trunc from 0x7bc67a to 0x316e5d (0x4a581c bytes) 000088[0] 1722238107.749841 write 0x3b99f2 thru 0x3baed5 (0x14e4 bytes) 000089[1] 1722238107.755097 mapwrite 0x44048a thru 0x441b9f (0x1716 bytes) 000090[0] 1722238107.777109 write 0x3c223c thru 0x3c3a25 (0x17ea bytes) 000091[1] 1722238107.784106 read 0x29ec12 thru 0x29f23c (0x62b bytes) 000092[1] 1722238107.786919 read 0x0de809 thru 0x0ded5c (0x554 bytes) 000093[1] 1722238107.791803 read 0x38eaf4 thru 0x38eccc (0x1d9 bytes) 000099[0] 1722238107.802331 trunc from 0x441ba0 to 0x387055 (0xbab4a bytes) 000103[1] 1722238107.823055 mapread 0x202bf3 thru 0x203287 (0x695 bytes) 000106[0] 1722238107.833183 trunc from 0x387056 to 0x1be5e3 (0x1c8a72 bytes) 000109[0] 1722238107.842779 mapwrite 0x513aab thru 0x5141b0 (0x706 bytes) 000112[0] 1722238107.870783 write 0x16877a thru 0x169fac (0x1833 bytes) 000113[1] 1722238107.876107 write 0x4364ee thru 0x4377da (0x12ed bytes) 000116[0] 1722238107.895144 write 0x0902fb thru 0x091630 (0x1336 bytes) 000120[0] 1722238107.905833 write 0x36076d thru 0x360c74 (0x508 bytes) 000123[1] 1722238107.907733 trunc from 0x5141b1 to 0x3192a7 (0x1faf09 bytes) 000124[0] 1722238107.922655 mapread 0x1d9312 thru 0x1da9f5 (0x16e4 bytes) 000128[0] 1722238107.937423 mapwrite 0x0e731a thru 0x0e7cdd (0x9c4 bytes) 000129[0] 1722238107.960164 mapread 0x194091 thru 0x194696 (0x606 bytes) 000130[1] 1722238107.964299 mapwrite 0x56597d thru 0x566b75 (0x11f9 bytes) 000132[0] 1722238107.989685 write 0x461152 thru 0x462b06 (0x19b5 bytes) 000133[0] 1722238107.996843 mapwrite 0x6f5a43 thru 0x6f5a86 (0x44 bytes) 000134[1] 1722238108.020104 read 0x03aec2 thru 0x03bc66 (0xda5 bytes) 000135[0] 1722238108.032290 write 0x0bdc74 thru 0x0bde02 (0x18f bytes) 000136[0] 1722238108.036317 write 0x5504eb thru 0x5517e5 (0x12fb bytes) 000137[1] 1722238108.041190 read 0x38f2db thru 0x3905f9 (0x131f bytes) 000139[0] 1722238108.051872 mapread 0x1f5a1f thru 0x1f6df5 (0x13d7 bytes) 000140[1] 1722238108.056929 trunc from 0x6f5a87 to 0x17a6e5 (0x57b3a1 bytes) 000141[0] 1722238108.068318 mapread 0x03ccee thru 0x03e7db (0x1aee bytes) 000142[1] 1722238108.077650 write 0x291c0a thru 0x292759 (0xb50 bytes) 000143[1] 1722238108.083620 mapread 0x19d4ee thru 0x19ea6b (0x157e bytes) 000144[0] 1722238108.090130 mapwrite 0x482825 thru 0x483fd1 (0x17ad bytes) 000145[0] 1722238108.121645 read 0x02a724 thru 0x02b8c8 (0x11a5 bytes) 000146[0] 1722238108.146462 mapwrite 0x4a6272 thru 0x4a674d (0x4dc bytes) 000147[1] 1722238108.171902 mapwrite 0x1e6463 thru 0x1e646f (0xd bytes) 000149[0] 1722238108.196533 read 0x44fe5b thru 0x450cfc (0xea2 bytes) 000150[0] 1722238108.199670 mapread 0x2b7860 thru 0x2b7a46 (0x1e7 bytes) 000151[1] 1722238108.203926 mapwrite 0x351dfc thru 0x352346 (0x54b bytes) 000152[1] 1722238108.223007 mapwrite 0x9a92ee thru 0x9aa92c (0x163f bytes) 000153[1] 1722238108.259692 write 0x6d49fd thru 0x6d69f3 (0x1ff7 bytes) 000155[1] 1722238108.267047 read 0x8f3d77 thru 0x8f472e (0x9b8 bytes) 000157[0] 1722238108.270675 mapread 0x3cd25e thru 0x3ceead (0x1c50 bytes) 000158[1] 1722238108.281087 mapwrite 0x73be16 thru 0x73d636 (0x1821 bytes) 000159[0] 1722238108.299262 write 0x59a29e thru 0x59ada9 (0xb0c bytes) 000160[0] 1722238108.304712 trunc from 0x9aa92d to 0x520530 (0x48a3fc bytes) 000161[1] 1722238108.315735 read 0x0d5483 thru 0x0d569f (0x21d bytes) 000162[1] 1722238108.324255 mapwrite 0x6893b5 thru 0x689ed8 (0xb24 bytes) truncating to largest ever: 0x9cd6bf 000164[0] 1722238108.372486 trunc from 0x689ed9 to 0x9cd6be (0x3437e6 bytes) 000166[1] 1722238108.385160 read 0x17409e thru 0x174abc (0xa1f bytes) 000171[1] 1722238108.402254 mapread 0x396c0e thru 0x3987f3 (0x1be6 bytes) 000172[1] 1722238108.406600 mapwrite 0x7f7bd6 thru 0x7f9876 (0x1ca1 bytes) 000173[0] 1722238108.428410 mapwrite 0x33c90e thru 0x33d79f (0xe92 bytes) 000174[0] 1722238108.461467 trunc from 0x9cd6bf to 0x2b9477 (0x714247 bytes) 000176[0] 1722238108.469654 mapread 0x105a81 thru 0x105c88 (0x208 bytes) 000178[1] 1722238108.474954 mapwrite 0x27fdb3 thru 0x280f18 (0x1166 bytes) 000179[1] 1722238108.503046 write 0x1af947 thru 0x1b13fb (0x1ab5 bytes) 000181[1] 1722238108.512065 trunc from 0x2b9478 to 0x956381 (0x69cf0a bytes) 000182[1] 1722238108.519647 mapread 0x4547ae thru 0x45530e (0xb61 bytes) 000183[0] 1722238108.524463 write 0x7c7662 thru 0x7c7c09 (0x5a8 bytes) 000184[1] 1722238108.533382 trunc from 0x956382 to 0x527fab (0x42e3d6 bytes) 000186[1] 1722238108.543461 write 0x8abd82 thru 0x8ac574 (0x7f3 bytes) 000187[1] 1722238108.544408 trunc from 0x8ac575 to 0x98c155 (0xdfbe1 bytes) 000188[1] 1722238108.550917 write 0x264535 thru 0x2652e3 (0xdaf bytes) 000189[1] 1722238108.553719 trunc from 0x98c156 to 0x735a6a (0x2566eb bytes) 000193[1] 1722238108.563854 mapread 0x07fa95 thru 0x0813b7 (0x1923 bytes) 000197[1] 1722238108.570211 read 0x141753 thru 0x143392 (0x1c40 bytes) 000199[0] 1722238108.588636 write 0x4d93aa thru 0x4db333 (0x1f8a bytes) 000200[0] 1722238108.602534 mapwrite 0x2fc1f6 thru 0x2fdeac (0x1cb7 bytes) 000202[0] 1722238108.624843 trunc from 0x735a6b to 0x83ef48 (0x1094de bytes) 000203[0] 1722238108.632711 write 0x9a9562 thru 0x9aabd8 (0x1677 bytes) 000204[1] 1722238108.633409 write 0x8b40d2 thru 0x8b4cdf (0xc0e bytes) 000205[1] 1722238108.638793 read 0x49e603 thru 0x49f5ad (0xfab bytes) 000206[0] 1722238108.646974 mapread 0x389fa5 thru 0x38b1dc (0x1238 bytes) 000211[1] 1722238108.664282 read 0x2ca8d3 thru 0x2cb55f (0xc8d bytes) 000212[1] 1722238108.670020 mapwrite 0x661f94 thru 0x663f13 (0x1f80 bytes) 000213[1] 1722238108.688815 read 0x8879b0 thru 0x888ade (0x112f bytes) 000214[1] 1722238108.695313 trunc from 0x9aabd9 to 0x1851fc (0x8259dc bytes) 000219[1] 1722238108.711022 write 0x036f67 thru 0x037a67 (0xb01 bytes) 000220[1] 1722238108.713439 write 0x8a8d12 thru 0x8a96ff (0x9ee bytes) 000221[0] 1722238108.718247 write 0x6d5163 thru 0x6d5785 (0x623 bytes) 000222[1] 1722238108.724991 mapread 0x2575ac thru 0x257787 (0x1dc bytes) 000223[0] 1722238108.732960 mapwrite 0x82ba49 thru 0x82d0d6 (0x168e bytes) 000224[1] 1722238108.753437 mapwrite 0x9c60e7 thru 0x9c7d12 (0x1c2c bytes) 000225[1] 1722238108.780299 read 0x9a87ef thru 0x9aa2e5 (0x1af7 bytes) 000226[0] 1722238108.783101 write 0x84d0d2 thru 0x84d46f (0x39e bytes) 000228[1] 1722238108.795855 write 0x59557c thru 0x596be8 (0x166d bytes) 000236[1] 1722238108.819887 trunc from 0x9c7d13 to 0x50c2ac (0x4bba66 bytes) 000239[1] 1722238108.830866 mapwrite 0x0245c4 thru 0x024675 (0xb2 bytes) 000241[0] 1722238108.846239 mapwrite 0x3e64c7 thru 0x3e69d2 (0x50c bytes) 000244[0] 1722238108.868111 mapread 0x20c801 thru 0x20e4f2 (0x1cf2 bytes) 000245[0] 1722238108.873816 write 0x565a3a thru 0x5664ae (0xa75 bytes) 000247[1] 1722238108.874454 write 0x7edc5f thru 0x7edf65 (0x307 bytes) 000249[1] 1722238108.881448 mapread 0x1bccf6 thru 0x1be8d3 (0x1bde bytes) 000251[0] 1722238108.885840 trunc from 0x7edf66 to 0x2d7cd3 (0x516292 bytes) 000253[1] 1722238108.899129 read 0x1683e9 thru 0x168661 (0x279 bytes) 000256[1] 1722238108.906058 trunc from 0x2d7cd4 to 0x244221 (0x93ab2 bytes) 000259[1] 1722238108.921418 write 0x5c1675 thru 0x5c303f (0x19cb bytes) 000262[1] 1722238108.925791 write 0x4a7fbd thru 0x4a9810 (0x1854 bytes) 000264[0] 1722238108.934918 write 0x50f666 thru 0x510a4c (0x13e7 bytes) 000265[0] 1722238108.942691 mapwrite 0x8fb1a4 thru 0x8fc43c (0x1299 bytes) 000267[0] 1722238108.974139 read 0x3ccb90 thru 0x3cdef2 (0x1363 bytes) 000268[0] 1722238108.994716 trunc from 0x8fc43d to 0x40a770 (0x4f1ccc bytes) 000271[0] 1722238109.004346 mapwrite 0x370628 thru 0x37087b (0x254 bytes) 000272[0] 1722238109.023459 mapread 0x1e5625 thru 0x1e70b2 (0x1a8e bytes) 000273[1] 1722238109.031089 trunc from 0x40a771 to 0x8a6697 (0x49bf27 bytes) 000275[0] 1722238109.039958 mapread 0x72eb58 thru 0x72ecef (0x198 bytes) 000277[1] 1722238109.048343 mapread 0x40bf1d thru 0x40d862 (0x1946 bytes) 000278[1] 1722238109.050847 trunc from 0x8a6698 to 0x091655 (0x815042 bytes) 000281[0] 1722238109.061082 mapwrite 0x2024d9 thru 0x203737 (0x125f bytes) 000283[0] 1722238109.084475 mapwrite 0x19e60e thru 0x19e9f2 (0x3e5 bytes) 000285[0] 1722238109.103490 read 0x040fb4 thru 0x0411e1 (0x22e bytes) 000286[1] 1722238109.114003 mapread 0x15b0c3 thru 0x15be55 (0xd93 bytes) 000287[1] 1722238109.119868 write 0x95f6ed thru 0x96118c (0x1aa0 bytes) 000288[0] 1722238109.126307 mapwrite 0x22db43 thru 0x22eaad (0xf6b bytes) 000289[1] 1722238109.151697 read 0x4a5c52 thru 0x4a780a (0x1bb9 bytes) 000290[0] 1722238109.174566 mapwrite 0x882f2e thru 0x883577 (0x64a bytes) 000291[0] 1722238109.209282 trunc from 0x96118d to 0x0631e7 (0x8fdfa5 bytes) truncating to largest ever: 0x9eeb80 000293[0] 1722238109.224340 trunc from 0x0631e8 to 0x9eeb7f (0x98b998 bytes) 000294[1] 1722238109.232371 mapread 0x77a835 thru 0x77b099 (0x865 bytes) 000297[0] 1722238109.248997 write 0x874951 thru 0x8758bc (0xf6c bytes) 000298[1] 1722238109.252973 write 0x077354 thru 0x078d78 (0x1a25 bytes) 000300[0] 1722238109.263932 mapwrite 0x1c1106 thru 0x1c1ddf (0xcda bytes) 000304[0] 1722238109.301776 write 0x55bf14 thru 0x55c6e5 (0x7d2 bytes) 000306[0] 1722238109.309362 write 0x14d175 thru 0x14f145 (0x1fd1 bytes) 000308[0] 1722238109.316817 read 0x5fcbf5 thru 0x5fd3b5 (0x7c1 bytes) 000309[1] 1722238109.320856 read 0x60e417 thru 0x60ff8e (0x1b78 bytes) 000312[1] 1722238109.339655 write 0x0d1c56 thru 0x0d2b72 (0xf1d bytes) 000313[1] 1722238109.345180 read 0x312632 thru 0x312822 (0x1f1 bytes) 000317[0] 1722238109.373918 read 0x2d67f1 thru 0x2d7ce2 (0x14f2 bytes) 000318[1] 1722238109.413021 mapread 0x827b1a thru 0x828df9 (0x12e0 bytes) 000320[1] 1722238109.416030 write 0x7b494b thru 0x7b637d (0x1a33 bytes) 000323[1] 1722238109.430732 trunc from 0x9eeb80 to 0x0d08d5 (0x91e2aa bytes) 000324[0] 1722238109.445351 mapread 0x02cc07 thru 0x02dd3b (0x1135 bytes) 000328[0] 1722238109.461669 read 0x04a4f9 thru 0x04bfd0 (0x1ad8 bytes) 000330[0] 1722238109.471856 write 0x7a4509 thru 0x7a64f8 (0x1ff0 bytes) 000331[1] 1722238109.475252 mapwrite 0x26552b thru 0x265efd (0x9d3 bytes) 000334[1] 1722238109.504450 mapwrite 0x85b568 thru 0x85d4da (0x1f73 bytes) 000335[0] 1722238109.529802 trunc from 0x85d4db to 0x41f04f (0x43e48b bytes) 000336[1] 1722238109.537643 mapwrite 0x4edcf0 thru 0x4ee547 (0x858 bytes) 000337[0] 1722238109.566870 read 0x0853b8 thru 0x086313 (0xf5c bytes) 000343[0] 1722238109.590317 write 0x29c4b3 thru 0x29d93f (0x148d bytes) 000344[1] 1722238109.598100 mapwrite 0x0588c7 thru 0x059522 (0xc5c bytes) 000345[1] 1722238109.623641 trunc from 0x4ee548 to 0x590ad6 (0xa258f bytes) 000346[0] 1722238109.628760 read 0x40733e thru 0x407559 (0x21c bytes) 000347[0] 1722238109.647454 mapread 0x13a0b1 thru 0x13ab17 (0xa67 bytes) 000348[0] 1722238109.656821 read 0x0950aa thru 0x096b00 (0x1a57 bytes) 000350[1] 1722238109.676851 mapread 0x0e2852 thru 0x0e4752 (0x1f01 bytes) 000352[0] 1722238109.681438 mapread 0x18a9eb thru 0x18bf5a (0x1570 bytes) 000355[0] 1722238109.682066 trunc from 0x590ad7 to 0x52494b (0x6c18b bytes) 000357[1] 1722238109.699291 trunc from 0x52494c to 0x38588d (0x19f0be bytes) 000359[1] 1722238109.707131 read 0x25e1c2 thru 0x25f6ec (0x152b bytes) 000361[0] 1722238109.721032 read 0x1f508c thru 0x1f631e (0x1293 bytes) 000363[1] 1722238109.726477 write 0x7fdf31 thru 0x7ff115 (0x11e5 bytes) 000364[1] 1722238109.732172 mapwrite 0x559e83 thru 0x55bd8e (0x1f0c bytes) 000365[1] 1722238109.757146 write 0x73ca25 thru 0x73e22d (0x1809 bytes) 000366[1] 1722238109.757691 trunc from 0x7ff116 to 0x6373a0 (0x1c7d75 bytes) 000367[0] 1722238109.763964 trunc from 0x6373a1 to 0x9df812 (0x3a8472 bytes) 000370[0] 1722238109.783513 mapread 0x146961 thru 0x146ddb (0x47b bytes) 000371[1] 1722238109.814908 mapwrite 0x1a294e thru 0x1a3bfb (0x12ae bytes) 000373[0] 1722238109.844971 write 0x7cf59b thru 0x7cf613 (0x79 bytes) 000374[0] 1722238109.851896 mapread 0x030034 thru 0x030cb1 (0xc7e bytes) 000375[1] 1722238109.856407 trunc from 0x9df813 to 0x32a726 (0x6b50ec bytes) 000376[0] 1722238109.870812 write 0x85d785 thru 0x85f37b (0x1bf7 bytes) 000378[0] 1722238109.880878 mapwrite 0x342b9d thru 0x342c2f (0x93 bytes) 000381[0] 1722238109.903219 trunc from 0x85f37c to 0x3909b2 (0x4ce9c9 bytes) 000383[1] 1722238109.913426 mapread 0x25a4d2 thru 0x25bc5e (0x178d bytes) 000385[0] 1722238109.923768 mapwrite 0x229676 thru 0x22a83a (0x11c5 bytes) 000386[1] 1722238109.943756 read 0x314c08 thru 0x314c08 (0x1 bytes) 000387[1] 1722238109.953466 read 0x090419 thru 0x091b45 (0x172d bytes) 000388[0] 1722238109.970329 trunc from 0x3909b3 to 0x8d8348 (0x547996 bytes) 000389[1] 1722238109.983972 write 0x9d325e thru 0x9d3f47 (0xcea bytes) 000394[1] 1722238109.999974 read 0x55cd1e thru 0x55eb83 (0x1e66 bytes) 000396[1] 1722238110.012125 read 0x9c8f67 thru 0x9c9766 (0x800 bytes) 000397[0] 1722238110.016975 mapread 0x827d3f thru 0x828fd4 (0x1296 bytes) 000398[0] 1722238110.023621 mapread 0x00ee59 thru 0x0101e1 (0x1389 bytes) 000400[0] 1722238110.033470 mapread 0x9beeb6 thru 0x9c05a8 (0x16f3 bytes) 000402[0] 1722238110.046223 mapwrite 0x02227a thru 0x0230d5 (0xe5c bytes) 000404[0] 1722238110.063733 trunc from 0x9d3f48 to 0x62b3d7 (0x3a8b70 bytes) 000406[0] 1722238110.069673 write 0x6942d6 thru 0x6947c4 (0x4ef bytes) 000407[0] 1722238110.070103 trunc from 0x6947c5 to 0x3ebd39 (0x2a8a8b bytes) 000408[1] 1722238110.075611 read 0x12b992 thru 0x12d82d (0x1e9c bytes) 000411[0] 1722238110.089140 mapread 0x017daa thru 0x019570 (0x17c7 bytes) 000413[0] 1722238110.096693 trunc from 0x3ebd3a to 0x2271e2 (0x1c4b57 bytes) 000418[1] 1722238110.108885 mapread 0x0a2be8 thru 0x0a352e (0x947 bytes) 000419[0] 1722238110.118461 mapread 0x1ec258 thru 0x1ec3f5 (0x19e bytes) 000420[1] 1722238110.120301 mapread 0x181890 thru 0x1820e4 (0x855 bytes) 000423[1] 1722238110.126068 read 0x1e39af thru 0x1e48c1 (0xf13 bytes) 000424[1] 1722238110.128278 mapwrite 0x585a60 thru 0x585b25 (0xc6 bytes) 000425[0] 1722238110.161303 write 0x48f74a thru 0x4903de (0xc95 bytes) 000426[0] 1722238110.168369 write 0x7d887f thru 0x7d89e7 (0x169 bytes) 000427[0] 1722238110.174839 mapread 0x6b0ba6 thru 0x6b26c8 (0x1b23 bytes) 000428[0] 1722238110.181548 trunc from 0x7d89e8 to 0x447e5e (0x390b89 bytes) 000429[1] 1722238110.185096 trunc from 0x447e5f to 0x7c4ad8 (0x37cc7a bytes) truncating to largest ever: 0x9f6e34 000430[1] 1722238110.194294 trunc from 0x7c4ad9 to 0x9f6e33 (0x23235b bytes) 000433[0] 1722238110.201906 mapwrite 0x23f44a thru 0x23f8ac (0x463 bytes) 000437[1] 1722238110.228513 read 0x3d0dcd thru 0x3d11d0 (0x404 bytes) 000438[1] 1722238110.233514 mapwrite 0x4d1db8 thru 0x4d3aa3 (0x1cec bytes) 000440[0] 1722238110.274046 mapread 0x3e2f27 thru 0x3e4d90 (0x1e6a bytes) 000441[1] 1722238110.285289 mapread 0x8587c1 thru 0x85920c (0xa4c bytes) 000443[1] 1722238110.291196 read 0x4dc9a8 thru 0x4ddf8c (0x15e5 bytes) 000445[0] 1722238110.294217 mapwrite 0x3b7664 thru 0x3b8698 (0x1035 bytes) 000446[0] 1722238110.324940 mapread 0x80101f thru 0x801d0b (0xced bytes) 000449[1] 1722238110.334296 mapread 0x5ce7e9 thru 0x5ce9a2 (0x1ba bytes) 000450[0] 1722238110.338964 mapread 0x1f6e75 thru 0x1f7c8d (0xe19 bytes) 000452[0] 1722238110.341075 trunc from 0x9f6e34 to 0x45742d (0x59fa06 bytes) 000453[0] 1722238110.349979 mapread 0x21d6b8 thru 0x21ecc3 (0x160c bytes) 000454[1] 1722238110.352938 mapwrite 0x0762bc thru 0x07829f (0x1fe4 bytes) 000457[1] 1722238110.377386 read 0x3ef6e4 thru 0x3ef9cf (0x2ec bytes) 000458[0] 1722238110.383873 write 0x182674 thru 0x1830b0 (0xa3d bytes) 000461[0] 1722238110.395606 trunc from 0x45742e to 0x14e567 (0x308ec6 bytes) 000462[0] 1722238110.401469 mapread 0x0095ac thru 0x0096ed (0x142 bytes) 000463[1] 1722238110.406171 mapread 0x0233db thru 0x0241a9 (0xdcf bytes) 000465[1] 1722238110.414259 read 0x0add24 thru 0x0afb0e (0x1deb bytes) 000466[1] 1722238110.421549 write 0x62e9be thru 0x630090 (0x16d3 bytes) 000467[1] 1722238110.427150 mapwrite 0x63f9cc thru 0x6401dd (0x812 bytes) 000468[1] 1722238110.464273 read 0x206a51 thru 0x206ccf (0x27f bytes) 000469[1] 1722238110.464527 read 0x2d57f1 thru 0x2d6606 (0xe16 bytes) 000470[0] 1722238110.464741 mapwrite 0x8cf121 thru 0x8d0cc1 (0x1ba1 bytes) 000471[1] 1722238110.492536 mapwrite 0x8c0faa thru 0x8c143a (0x491 bytes) 000472[1] 1722238110.515364 read 0x2eaef9 thru 0x2ebe4a (0xf52 bytes) 000474[1] 1722238110.521105 write 0x9b9d35 thru 0x9bab5b (0xe27 bytes) 000475[1] 1722238110.523603 write 0x5f87bc thru 0x5f9aa6 (0x12eb bytes) 000478[1] 1722238110.550464 read 0x0497ad thru 0x04a904 (0x1158 bytes) 000479[1] 1722238110.567748 mapread 0x88d853 thru 0x88de89 (0x637 bytes) 000480[1] 1722238110.570797 read 0x4a2b40 thru 0x4a30ad (0x56e bytes) 000481[1] 1722238110.573755 mapwrite 0x4f3657 thru 0x4f54a0 (0x1e4a bytes) 000482[1] 1722238110.588605 trunc from 0x9bab5c to 0x13385e (0x8872fd bytes) 000483[0] 1722238110.602031 read 0x11168f thru 0x1119a0 (0x312 bytes) 000485[0] 1722238110.607305 write 0x0129f3 thru 0x014996 (0x1fa4 bytes) 000486[0] 1722238110.614518 write 0x258652 thru 0x25a2d5 (0x1c84 bytes) 000487[0] 1722238110.615140 read 0x1be4fa thru 0x1bfe85 (0x198c bytes) 000489[0] 1722238110.618210 mapwrite 0x3e9741 thru 0x3ea744 (0x1004 bytes) 000490[0] 1722238110.646294 read 0x3bce6b thru 0x3be232 (0x13c8 bytes) 000491[0] 1722238110.649237 mapread 0x3b0a5d thru 0x3b201c (0x15c0 bytes) 000493[1] 1722238110.650639 write 0x00d9b1 thru 0x00f1cd (0x181d bytes) 000495[1] 1722238110.661321 trunc from 0x3ea745 to 0x7e4604 (0x3f9ec0 bytes) 000496[0] 1722238110.670636 mapread 0x3b8cbc thru 0x3ba35b (0x16a0 bytes) 000497[0] 1722238110.680393 read 0x28abea thru 0x28ade0 (0x1f7 bytes) 000500[0] 1722238110.704107 read 0x3e8a4e thru 0x3e95de (0xb91 bytes) All operations completed A-OK! PASS 16b (5s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 03:28:31 (1722238111) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 03:28:32 (1722238112) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0847268 s, 24.8 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0615875 s, 34.1 MB/s ldlm.namespaces.MGC192.168.203.107@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aa4c3800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b3e32800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800aa4c3800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b3e32800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800aa4c3800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b3e32800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.62011 s, 40.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.606408 s, 173 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.526514 s, 199 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.13375 s, 92.5 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.78664 s, 58.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.792271 s, 132 MB/s PASS 16d (10s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 03:28:42 (1722238122) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0557723 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030897 s, 33.9 MB/s PASS 16e (2s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 03:28:44 (1722238124) TIMEOUT PASS 16f (21s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 03:29:05 (1722238145) TIMEOUT PASS 16g (22s) == sanityn test 16i: read after truncate file ============ 03:29:27 (1722238167) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (1s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 03:29:28 (1722238168) fail_loc=0x8000030a PASS 17 (4s) == sanityn test 18: mmap sanity check =========================================================================================== 03:29:32 (1722238172) mmap test1: basic mmap operation (PASS, 0.014495s) mmap test2: MAP_PRIVATE not write back (PASS, 0.010917s) mmap test3: concurrent mmap ops on two nodes (PASS, 6.5323s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.069s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.021369s) mmap test6: check mmap write/read content on two nodes (PASS, 0.027554s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0106s) PASS 18 (12s) == sanityn test 19: test concurrent uncached read races ========================================================================= 03:29:44 (1722238184) oleg307-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg307-client: oleg307-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 03:29:45 (1722238185) PASS 20 (1s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 03:29:46 (1722238186) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) == sanityn test 23: others should see updated atime while another read============================================================== 03:29:48 (1722238188) now is 1722238188 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6837 new atime is 1722238249 PASS 23 (62s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 03:30:50 (1722238250) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 12288 7526400 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 12288 7526400 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.3K 376 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.2K 369 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 376 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 369 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 12.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118040 376 117664 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235697 369 235328 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 12288 7526400 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.3K 376 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.2K 369 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118040 376 117664 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre[OST:1] filesystem_summary: 235697 369 235328 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118040 376 117664 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235697 369 235328 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 12288 7526400 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 12288 7526400 1% /mnt/lustre2 PASS 24a (2s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 03:30:52 (1722238252) PASS 24b (1s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 03:30:53 (1722238253) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (2s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 03:30:55 (1722238255) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (0s) == sanityn test 26a: allow mtime to get older ============ 03:30:55 (1722238255) utime: good mknod times 1722238255 <= 1722238256 <= 1722238256 for /mnt/lustre/f26a utime: good mknod times 1722238255 <= 1722238256 <= 1722238256 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (3s) == sanityn test 26b: sync mtime between ost and mds ====== 03:30:58 (1722238258) PASS 26b (3s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 03:31:02 (1722238262) 0 PASS 30 (3s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 03:31:05 (1722238265) fail_loc=0x314 PASS 31a (3s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 03:31:08 (1722238268) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg307-server mds-ost sync done. sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.203.107@tcp:/lustre 7542784 13312 7525376 1% /mnt/lustre2 PASS 31b (21s) == sanityn test 31r: open-rename(replace) race =========== 03:31:29 (1722238289) -rw-r--r-- 1 root root 0 Jul 29 03:31 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (5s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 03:31:34 (1722238294) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 03:31:35 (1722238295) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 03:31:36 (1722238296) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (0s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 03:31:37 (1722238297) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (0s) == sanityn test 34: no lock timeout under IO ============= 03:31:37 (1722238297) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg307-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa4c3800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b3e32800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800aa4c3800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8800b3e32800.ost_server_uuid in FULL state after 0 sec oleg307-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa4c3800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b3e32800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800aa4c3800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800b3e32800.ost_server_uuid in FULL state after 0 sec oleg307-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa4c3800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b3e32800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800aa4c3800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8800b3e32800.ost_server_uuid in IDLE state after 0 sec oleg307-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa4c3800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b3e32800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800aa4c3800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800b3e32800.ost_server_uuid in FULL state after 0 sec oleg307-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa4c3800.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b3e32800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800aa4c3800.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8800b3e32800.ost_server_uuid in IDLE state after 0 sec oleg307-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa4c3800.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b3e32800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800aa4c3800.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800b3e32800.ost_server_uuid in IDLE state after 0 sec PASS 34 (71s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 03:32:48 (1722238368) fail_loc=0x80000317 Race attempt 0 Wait for 15633 15772 for 60 sec... total: 4000 open/close in 8.80 seconds: 454.68 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.203.107@tcp:/lustre 7.2G 14M 7.2G 1% /mnt/lustre 192.168.203.107@tcp:/lustre 7.2G 14M 7.2G 1% /mnt/lustre2 PASS 35 (63s) == sanityn test 36: handle ESTALE/open-unlink correctly == 03:33:51 (1722238431) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg307-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.28518 s, 184 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6837 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.273966 s, 191 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6837 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.259672 s, 202 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6837 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.283766 s, 185 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6837 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.236671 s, 222 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6837 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.245997 s, 213 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6837 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.242991 s, 216 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6837 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.314068 s, 167 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6837 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.376923 s, 139 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6837 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7524352) after_dd(7471104) after(7522304) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.292111 s, 179 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6837 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7522304) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.300289 s, 175 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6837 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7524352) after_dd(7473152) after(7524352) PASS 36 (228s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 03:37:39 (1722238659) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6837 - create 8885 (time 1722238670.48 total 10.00 last 888.40) total: 10000 create in 11.33 seconds: 882.45 ops/second PASS 37 (14s) == sanityn test 39a: test from 11063 ============================================================================================ 03:37:53 (1722238673) repeat after cancel_lru_locks PASS 39a (2s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 03:37:55 (1722238675) repeat after cancel_lru_locks PASS 39b (2s) == sanityn test 39c: check truncate mtime update ================================================================================ 03:37:57 (1722238677) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 03:38:00 (1722238680) fail_loc=0x411 fail_loc=0 PASS 39d (1s) == sanityn test 40a: pdirops: create vs others ======================================================================== 03:38:01 (1722238681) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (8s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 03:38:09 (1722238689) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (7s) == sanityn test 40c: pdirops: link and others ======================================================================== 03:38:16 (1722238696) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (7s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 03:38:23 (1722238703) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (8s) == sanityn test 40e: pdirops: rename and others ======================================================================== 03:38:31 (1722238711) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (6s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 03:38:37 (1722238717) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (5s) == sanityn test 41b: pdirops: create vs create ======================================================================== 03:38:42 (1722238722) open: File exists Conflict PASS 41b (4s) == sanityn test 41c: pdirops: create vs link ======================================================================== 03:38:46 (1722238726) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (5s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 03:38:51 (1722238731) Conflict PASS 41d (4s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 03:38:55 (1722238735) Conflict PASS 41e (4s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 03:38:59 (1722238739) Conflict PASS 41f (4s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 03:39:03 (1722238743) Conflict PASS 41g (4s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 03:39:07 (1722238747) Conflict PASS 41h (4s) == sanityn test 41i: reint_open: create vs create ======== 03:39:12 (1722238752) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (651s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 03:50:03 (1722239403) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (4s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 03:50:07 (1722239407) open: File exists Conflict PASS 42b (4s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 03:50:11 (1722239411) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (4s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 03:50:16 (1722239416) Conflict PASS 42d (4s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 03:50:20 (1722239420) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (4s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 03:50:24 (1722239424) Conflict PASS 42f (4s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 03:50:28 (1722239428) Conflict PASS 42g (4s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 03:50:32 (1722239432) Conflict PASS 42h (4s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 03:50:36 (1722239436) PASS 43a (11s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 03:50:47 (1722239447) Conflict PASS 43b (4s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 03:50:51 (1722239451) Conflict PASS 43c (4s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 03:50:55 (1722239455) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (4s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 03:50:59 (1722239459) Conflict PASS 43e (4s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 03:51:03 (1722239463) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (5s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 03:51:08 (1722239468) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (4s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 03:51:12 (1722239472) Conflict PASS 43h (5s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 03:51:17 (1722239477) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (0s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 03:51:17 (1722239477) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (34s) == sanityn test 43k: unlink vs create ==================== 03:51:51 (1722239511) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (376s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 03:58:07 (1722239887) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Mon Jul 29 03:58:10 EDT 2024 PASS 44a (4s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 03:58:11 (1722239891) open: File exists Conflict PASS 44b (4s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 03:58:15 (1722239895) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (5s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 03:58:20 (1722239900) Conflict PASS 44d (4s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 03:58:24 (1722239904) Conflict PASS 44e (5s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 03:58:29 (1722239909) Conflict PASS 44f (4s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 03:58:33 (1722239913) Conflict PASS 44g (5s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 03:58:38 (1722239918) Conflict PASS 44h (4s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 03:58:42 (1722239922) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 03:58:43 (1722239923) PASS 45a (24s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 03:59:07 (1722239947) Conflict PASS 45b (5s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 03:59:12 (1722239952) Conflict PASS 45c (4s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 03:59:16 (1722239956) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (5s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 03:59:21 (1722239961) Conflict PASS 45e (4s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 03:59:25 (1722239965) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (5s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 03:59:30 (1722239970) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (4s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 03:59:34 (1722239974) Conflict PASS 45h (4s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 03:59:38 (1722239978) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (1s) == sanityn test 45j: read vs rename ====================== 03:59:39 (1722239979) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (374s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 04:05:53 (1722240353) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (4s) == sanityn test 46b: pdirops: link vs create ======================================================================== 04:05:57 (1722240357) open: File exists Conflict PASS 46b (5s) == sanityn test 46c: pdirops: link vs link ======================================================================== 04:06:02 (1722240362) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (4s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 04:06:06 (1722240366) Conflict PASS 46d (4s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 04:06:10 (1722240370) Conflict PASS 46e (4s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 04:06:14 (1722240374) Conflict PASS 46f (5s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 04:06:19 (1722240379) Conflict PASS 46g (4s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 04:06:23 (1722240383) Conflict PASS 46h (4s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 04:06:27 (1722240387) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (0s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 04:06:27 (1722240387) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 04:06:28 (1722240388) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 04:06:29 (1722240389) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 04:06:30 (1722240390) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (0s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 04:06:30 (1722240390) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 04:06:31 (1722240391) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 04:06:32 (1722240392) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 04:06:33 (1722240393) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.00569551 s, 1.8 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (6s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 04:06:39 (1722240399) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00300784 s, 52.9 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3175: kill: (24615) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 04:06:43 (1722240403) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00203828 s, 502 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011406 s, 91.9 MB/s 3145728 PASS 51b (13s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 04:06:56 (1722240416) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00329314 s, 311 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 04:07:02 (1722240422) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105857 s, 99.1 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (2s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 04:07:04 (1722240424) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2430 0x97e 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2430 0x97e 0 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 04:07:08 (1722240428) llite.lustre-ffff8800aa4c3800.xattr_cache=1 llite.lustre-ffff8800b3e32800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504963 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-29 04:07:08.000000000 -0400 Modify: 2024-07-29 04:07:08.000000000 -0400 Change: 2024-07-29 04:07:08.000000000 -0400 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504966 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-29 04:07:14.000000000 -0400 Modify: 2024-07-29 04:07:14.000000000 -0400 Change: 2024-07-29 04:07:14.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504969 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-29 04:07:19.000000000 -0400 Modify: 2024-07-29 04:07:19.000000000 -0400 Change: 2024-07-29 04:07:19.000000000 -0400 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-29 04:07:19.000000000 -0400 Modify: 2024-07-29 04:07:19.000000000 -0400 Change: 2024-07-29 04:07:19.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-29 04:07:24.000000000 -0400 Modify: 2024-07-29 04:07:25.000000000 -0400 Change: 2024-07-29 04:07:25.000000000 -0400 Birth: - File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504972 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-29 04:07:24.000000000 -0400 Modify: 2024-07-29 04:07:25.000000000 -0400 Change: 2024-07-29 04:07:25.000000000 -0400 Birth: - PASS 54 (23s) == sanityn test 55a: rename vs unlink target dir ========= 04:07:31 (1722240451) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 04:07:38 (1722240458) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 04:07:45 (1722240465) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 04:07:57 (1722240477) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) == sanityn test 60: Verify data_version behaviour ======== 04:08:06 (1722240486) PASS 60 (1s) == sanityn test 70a: cd directory && rm directory ======== 04:08:07 (1722240487) PASS 70a (2s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 04:08:09 (1722240489) PASS 70b (1s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 04:08:11 (1722240491) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00248942 s, 26.3 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (0s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 04:08:11 (1722240491) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00412695 s, 15.9 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 04:08:12 (1722240492) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 04:08:13 (1722240493) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00374323 s, 17.5 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 04:08:14 (1722240494) llite.lustre-ffff8800aa4c3800.xattr_cache=1 llite.lustre-ffff8800b3e32800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (1s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 04:08:15 (1722240495) llite.lustre-ffff8800aa4c3800.xattr_cache=1 llite.lustre-ffff8800b3e32800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (2s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 04:08:17 (1722240497) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00221298 s, 463 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00524934 s, 195 kB/s 3267: get lock2 3268: get lock1 3268: done 3268: get lock2 3267: done 3267: get lock1 fcntl cmd 7 failed: Resource deadlock avoided 3267: cannot get lock1: Resource deadlock avoided fcntl cmd 7 failed: Resource deadlock avoided 3268: cannot get lock2: Resource deadlock avoided 3268: exit rc=0 3267: done 3267: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 04:08:23 (1722240503) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0141823 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00674818 s, 155 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3743: 3901 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (6s) == sanityn test 76: Verify MDT open_files listing ======== 04:08:29 (1722240509) opening files: ........ closing files: ........ PASS 76 (15s) == sanityn test 77a: check FIFO NRS policy =============== 04:08:44 (1722240524) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.109378 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0091359 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142551 s, 73.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192062 s, 54.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0238742 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390584 s, 26.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0369206 s, 28.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0328086 s, 32.0 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0397017 s, 26.4 MB/s , 0.035093 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328574 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.033604 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0413919 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119584 s, 87.7 MB/s , 0.0318606 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.035344 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386404 s, 27.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3e32800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0021144 s, 496 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101206 s, 104 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110712 s, 94.7 MB/s 1048576 bytes (1.0 MB) copied, 0.00984522 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112295 s, 93.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011279 s, 93.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0106865 s, 98.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010933 s, 95.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124936 s, 83.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118974 s, 88.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135458 s, 77.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0126509 s, 82.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0130341 s, 80.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138182 s, 75.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129437 s, 81.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167011 s, 62.8 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) == sanityn test 77b: check CRR-N NRS policy ============== 04:08:46 (1722240526) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0937828 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00842034 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00681401 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00841331 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231365 s, 45.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014618 s, 71.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134101 s, 78.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0182049 s, 57.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186627 s, 56.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0072401 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.040711 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0421375 s, 24.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.043351 s, 24.2 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3e32800.lock_unused_count=1 1048576 bytes (1.0 MB) copied, 0.0491338 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0334157 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321003 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0543296 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102388 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00758193 s, 138 MB/s 1048576 bytes (1.0 MB) copied, 0.00749424 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00954314 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00955304 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00986582 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00916516 s, 114 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00235103 s, 446 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0105998 s, 98.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105787 s, 99.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0085601 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137926 s, 76.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119361 s, 87.8 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3e32800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00153437 s, 683 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130056 s, 80.6 MB/s , 0.0094498 s, 111 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0961264 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00861053 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00817783 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00817819 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162792 s, 64.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120419 s, 87.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206292 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124845 s, 84.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00798653 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114631 s, 91.5 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3e32800.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0385574 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290889 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319346 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0438525 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0582579 s, 18.0 MB/s 1048576 bytes (1.0 MB) copied, 0.043927 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0639669 s, 16.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3e32800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.009519 s, 110 MB/s , 0.00160279 s, 654 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00180031 s, 582 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00662068 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00756743 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00880946 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00647582 s, 162 MB/s 1048576 bytes (1.0 MB) copied, 0.00640108 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00655385 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00676789 s, 155 MB/s , 0.00771774 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00844625 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00861246 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00351564 s, 298 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104144 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00140274 s, 748 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) == sanityn test 77c: check ORR NRS policy ================ 04:08:49 (1722240529) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.149331 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123396 s, 85.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111703 s, 93.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174372 s, 60.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256666 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0314879 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277148 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124047 s, 84.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233727 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280065 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0257553 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194639 s, 53.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0170069 s, 61.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0391699 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335935 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265309 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378977 s, 27.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3e32800.lock_unused_count=2 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3e32800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00874439 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00170757 s, 614 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00190101 s, 552 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984557 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102209 s, 103 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010454 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150011 s, 69.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00158867 s, 660 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120696 s, 86.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141368 s, 74.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0143906 s, 72.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0112291 s, 93.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0131177 s, 79.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113204 s, 92.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0117861 s, 89.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0114301 s, 91.7 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.105925 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00892282 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111037 s, 94.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158187 s, 66.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259989 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240451 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146358 s, 71.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226675 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130629 s, 80.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143355 s, 73.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0314336 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0468484 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0373558 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447551 s, 23.4 MB/s , 0.041948 s, 25.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3e32800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328759 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372231 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00674216 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107542 s, 97.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00881903 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108708 s, 96.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104122 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0095071 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100445 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00658091 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011223 s, 93.4 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3e32800.lock_unused_count=2 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010661 s, 98.4 MB/s 1048576 bytes (1.0 MB) copied, 0.012962 s, 80.9 MB/s 1048576 bytes (1.0 MB) copied, 0.00250955 s, 418 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0103124 s, 102 MB/s 1048576 bytes (1.0 MB) copied, 0.00593463 s, 177 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191294 s, 54.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00129929 s, 807 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0955142 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00816145 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0087612 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184708 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200975 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206871 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211122 s, 49.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3e32800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0377801 s, 27.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0320704 s, 32.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457405 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0483731 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423897 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.053605 s, 19.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0521407 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388407 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.042174 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0450935 s, 23.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100839 s, 104 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015544 s, 67.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163414 s, 64.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106109 s, 98.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0150089 s, 69.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0112066 s, 93.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018061 s, 58.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182693 s, 57.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137601 s, 76.2 MB/s 1048576 bytes (1.0 MB) copied, 0.017425 s, 60.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.01496 s, 70.1 MB/s , 0.0148557 s, 70.6 MB/s , 0.014857 s, 70.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0153496 s, 68.3 MB/s , 0.0167378 s, 62.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152891 s, 68.6 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3e32800.lock_unused_count=6 removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (4s) == sanityn test 77d: check TRR nrs policy ================ 04:08:53 (1722240533) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.113796 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00717979 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0087182 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00934121 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269301 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272527 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259028 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388133 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238214 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294077 s, 35.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281058 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0425749 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0313533 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0357831 s, 29.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3e32800.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0343544 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014197 s, 73.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010252 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106062 s, 98.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00706745 s, 148 MB/s 1048576 bytes (1.0 MB) copied, 0.00949076 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00855211 s, 123 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0102458 s, 102 MB/s , 0.00331776 s, 316 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00860301 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00838278 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00919077 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00869339 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00141368 s, 742 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00922496 s, 114 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3e32800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122184 s, 85.8 MB/s , 0.0106889 s, 98.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109164 s, 96.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00348317 s, 301 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0970959 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00713108 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00832114 s, 126 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258402 s, 40.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0266791 s, 39.3 MB/s 1048576 bytes (1.0 MB) copied, 0.026941 s, 38.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0213742 s, 49.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238637 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218606 s, 48.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0258556 s, 40.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196518 s, 53.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155491 s, 67.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0131222 s, 79.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190728 s, 55.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025098 s, 41.8 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3e32800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284288 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107419 s, 97.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00898987 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00822401 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00829427 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00367187 s, 286 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00858859 s, 122 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00941489 s, 111 MB/s , 0.00919785 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968648 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00157157 s, 667 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00888748 s, 118 MB/s , 0.00318654 s, 329 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.011952 s, 87.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0112944 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118671 s, 88.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0133568 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00413556 s, 254 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3e32800.lock_unused_count=1 removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.109049 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128608 s, 81.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130696 s, 80.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207824 s, 50.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283509 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331348 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447122 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0520506 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.048536 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0546629 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0476519 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0544764 s, 19.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0556596 s, 18.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0569599 s, 18.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0706564 s, 14.8 MB/s 1048576 bytes (1.0 MB) copied, 0.06332 s, 16.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0616606 s, 17.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3e32800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3e32800.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00906321 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00191283 s, 548 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00179405 s, 584 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00710383 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143171 s, 73.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00188054 s, 558 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00147664 s, 710 MB/s , 0.00150018 s, 699 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00183587 s, 571 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00162557 s, 645 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0013853 s, 757 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00217864 s, 481 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00161886 s, 648 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00149219 s, 703 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00177978 s, 589 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00147768 s, 710 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (4s) == sanityn test 77e: check TBF NID nrs policy ============ 04:08:57 (1722240537) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.203.7@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0944386 s, 178 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114254 s, 91.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132091 s, 79.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261014 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272023 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276272 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0429656 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0484776 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0483917 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0453287 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0501879 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0547 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0422033 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0484286 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0447137 s, 23.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0526924 s, 19.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457883 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00726666 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00678408 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00788459 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103321 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00937574 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00843521 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0094169 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00140553 s, 746 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103326 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00961862 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0095728 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101639 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106109 s, 98.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117321 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104389 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120842 s, 86.8 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.086122 s, 195 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013122 s, 79.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133725 s, 78.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0422852 s, 24.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0367353 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354518 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0438838 s, 23.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0480916 s, 21.8 MB/s , 0.0447447 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464772 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0465057 s, 22.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0474978 s, 22.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0502299 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0479107 s, 21.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0442111 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.047905 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0506244 s, 20.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100895 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00800716 s, 131 MB/s , 0.0101501 s, 103 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00766529 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00813896 s, 129 MB/s 1048576 bytes (1.0 MB) copied, 0.00856947 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00906983 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010104 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105432 s, 99.5 MB/s 1048576 bytes (1.0 MB) copied, 0.00986752 s, 106 MB/s 1048576 bytes (1.0 MB) copied, 0.00994049 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110599 s, 94.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104453 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114111 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113507 s, 92.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127954 s, 81.9 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0947729 s, 177 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104901 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103634 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0309636 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273784 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332595 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327555 s, 32.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0488355 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338116 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0416962 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390697 s, 26.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0435211 s, 24.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0418508 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386894 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0536146 s, 19.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0392441 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0435476 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00595333 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00775343 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00618163 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121763 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00798119 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0079828 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948457 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00833662 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0083599 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101824 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120135 s, 87.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114439 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115957 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123234 s, 85.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0126463 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125309 s, 83.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3e32800.lock_unused_count=1 removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0942537 s, 178 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00772236 s, 136 MB/s 1048576 bytes (1.0 MB) copied, 0.0101937 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211619 s, 49.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102592 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164123 s, 63.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260318 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212251 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274735 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206253 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282598 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251376 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0334751 s, 31.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282913 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.036217 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126997 s, 82.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3e32800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3e32800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243802 s, 43.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103995 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00169718 s, 618 MB/s 1048576 bytes (1.0 MB) copied, 0.0103666 s, 101 MB/s , 0.00895228 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848734 s, 124 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00875823 s, 120 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00689887 s, 152 MB/s , 0.00435153 s, 241 MB/s 1048576 bytes (1.0 MB) copied, 0.00294432 s, 356 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139483 s, 75.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142387 s, 73.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00170007 s, 617 MB/s 1048576 bytes (1.0 MB) copied, 0.00183473 s, 572 MB/s 1048576 bytes (1.0 MB) copied, 0.00770493 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134852 s, 77.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00147512 s, 711 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3e32800.lock_unused_count=1 removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (6s) == sanityn test 77f: check TBF JobID nrs policy ========== 04:09:03 (1722240543) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.124309 s, 135 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00713076 s, 147 MB/s 1048576 bytes (1.0 MB) copied, 0.00722862 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0199785 s, 52.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0220345 s, 47.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0167319 s, 62.7 MB/s , 0.0218542 s, 48.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174987 s, 59.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124948 s, 83.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0244905 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026581 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0314463 s, 33.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0230615 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214443 s, 48.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187052 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147173 s, 71.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0252123 s, 41.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00704147 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00689924 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00957423 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00922353 s, 114 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00950835 s, 110 MB/s , 0.00762459 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00883076 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0084499 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00882327 s, 119 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102595 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0097489 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00926108 s, 113 MB/s 1048576 bytes (1.0 MB) copied, 0.011285 s, 92.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00955435 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00875257 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115135 s, 91.1 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0901936 s, 186 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00734531 s, 143 MB/s , 0.00741538 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112739 s, 93.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144903 s, 72.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0113983 s, 92.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00904904 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138697 s, 75.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119248 s, 87.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00770421 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333762 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253475 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00970411 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203554 s, 51.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0376661 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0340647 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0396448 s, 26.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00659708 s, 159 MB/s , 0.00454299 s, 231 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00752244 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00890765 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00751156 s, 140 MB/s , 0.00562618 s, 186 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.009636 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101118 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00908573 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104658 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00114744 s, 914 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00727116 s, 144 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00943608 s, 111 MB/s , 0.0105397 s, 99.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00986764 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117479 s, 89.3 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0846125 s, 198 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107563 s, 97.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157822 s, 66.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151959 s, 69.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178039 s, 58.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013351 s, 78.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217182 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183128 s, 57.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101918 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0312546 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271565 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015264 s, 68.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315756 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256323 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197466 s, 53.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0230552 s, 45.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00965349 s, 109 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3e32800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3e32800.lock_unused_count=1 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0080115 s, 131 MB/s , 0.00801137 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00744342 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105323 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00189881 s, 552 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00797232 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.007753 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00842798 s, 124 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00594179 s, 176 MB/s , 0.0121528 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00736614 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129311 s, 81.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00164942 s, 636 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105517 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00130203 s, 805 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00124782 s, 840 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0995555 s, 169 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00749121 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141645 s, 74.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187127 s, 56.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210927 s, 49.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185656 s, 56.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0420408 s, 24.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0313991 s, 33.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0429542 s, 24.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0305839 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0424464 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0376352 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0463398 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0463073 s, 22.6 MB/s , 0.0353198 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0339035 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0453438 s, 23.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3e32800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3e32800.lock_unused_count=7 running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00574693 s, 182 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00150007 s, 699 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00148858 s, 704 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00715396 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00880616 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0097468 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00893527 s, 117 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0015552 s, 674 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104009 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0017214 s, 609 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118013 s, 88.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0103791 s, 101 MB/s 1048576 bytes (1.0 MB) copied, 0.0113642 s, 92.3 MB/s 1048576 bytes (1.0 MB) copied, 0.00478827 s, 219 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120122 s, 87.3 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3e32800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153034 s, 68.5 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (5s) == sanityn test 77g: Change TBF type directly ============ 04:09:08 (1722240548) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (3s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 04:09:11 (1722240551) oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg307-client: oleg307-server: ssh exited with exit code 2 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg307-client: oleg307-server: ssh exited with exit code 12 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg307-client: oleg307-server: ssh exited with exit code 12 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg307-client: oleg307-server: ssh exited with exit code 12 PASS 77h (3s) == sanityn test 77i: Change rank of TBF rule ============= 04:09:14 (1722240554) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (5s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 04:09:19 (1722240559) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0968825 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111763 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125273 s, 83.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143774 s, 72.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106919 s, 98.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189535 s, 55.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0100445 s, 104 MB/s , 0.00720249 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016044 s, 65.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141034 s, 74.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364571 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268326 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235002 s, 44.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0399144 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0433188 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279777 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281038 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00697297 s, 150 MB/s , 0.00919996 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00704236 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00800154 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114744 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00856031 s, 122 MB/s , 0.0089744 s, 117 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00967554 s, 108 MB/s 1048576 bytes (1.0 MB) copied, 0.00947381 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103702 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110989 s, 94.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0102194 s, 103 MB/s , 0.0103376 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103498 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109079 s, 96.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120091 s, 87.3 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93962 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4281 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (52s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 04:10:11 (1722240611) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.640271 s, 26.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.170098 s, 6.2 MB/s , 0.160845 s, 6.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.180865 s, 5.8 MB/s , 0.17909 s, 5.9 MB/s , 0.18271 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.187751 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.175832 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.184358 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.36968 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.382261 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.391725 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.385838 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.386536 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.402123 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.402744 s, 2.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.388754 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158482 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157602 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157688 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157116 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157505 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157779 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158099 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157831 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159117 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159818 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158249 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159028 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158743 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159029 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158734 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.161786 s, 6.5 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8286 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.968 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.443596 s, 37.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00827812 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0296954 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0195598 s, 53.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0351141 s, 29.9 MB/s , 0.0203748 s, 51.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0385112 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215379 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189377 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.366427 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.36511 s, 2.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.381769 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.368585 s, 2.8 MB/s , 0.376202 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.383944 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.383305 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.390332 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151242 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149942 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151498 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150393 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151774 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152362 s, 6.9 MB/s , 0.152301 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151817 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151938 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.153178 s, 6.8 MB/s 1048576 bytes (1.0 MB) copied, 0.153187 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153516 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153803 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154599 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154112 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155697 s, 6.7 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8292 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9643 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (129s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 04:12:20 (1722240740) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.194933 s, 86.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199368 s, 52.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178611 s, 58.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0232545 s, 45.1 MB/s , 0.032493 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271835 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202562 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326189 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286284 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0703344 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0859884 s, 12.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0751664 s, 14.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0832071 s, 12.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.079565 s, 13.2 MB/s , 0.0879278 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0825647 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0992443 s, 10.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0068021 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00829856 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00964469 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00826817 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00362924 s, 289 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008626 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00886437 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00596958 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105519 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010535 s, 99.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00759574 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114195 s, 91.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012536 s, 83.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119915 s, 87.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119518 s, 87.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127949 s, 82.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93676 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73151 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.203.7@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.203.7@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.190817 s, 87.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239223 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227841 s, 46.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269349 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02929 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109122 s, 96.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0363338 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0298089 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365701 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276397 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111859 s, 93.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.112937 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.107116 s, 9.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.205058 s, 5.1 MB/s , 0.198358 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.314937 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.293506 s, 3.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00868107 s, 121 MB/s 1048576 bytes (1.0 MB) copied, 0.00780548 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00518236 s, 202 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00755811 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00903297 s, 116 MB/s 1048576 bytes (1.0 MB) copied, 0.00954159 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00748332 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00718188 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105111 s, 99.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0094749 s, 111 MB/s 1048576 bytes (1.0 MB) copied, 0.00883047 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00884975 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00930515 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00949858 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00964396 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104482 s, 100 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92766 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73514 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.203.7@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.149291 s, 112 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017706 s, 59.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160274 s, 65.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324538 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217209 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0401034 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.026756 s, 39.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0387307 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0313398 s, 33.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0587415 s, 17.9 MB/s , 0.0492909 s, 21.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0794039 s, 13.2 MB/s , 0.0722856 s, 14.5 MB/s , 0.0717412 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0799103 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0725691 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0880348 s, 11.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00370419 s, 283 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00420709 s, 249 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00152336 s, 688 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00138451 s, 757 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102587 s, 102 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0100871 s, 104 MB/s , 0.0152855 s, 68.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00959254 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00237041 s, 442 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106752 s, 98.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0161357 s, 65.0 MB/s , 0.0155462 s, 67.4 MB/s , 0.00230728 s, 454 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00211485 s, 496 MB/s , 0.00477629 s, 220 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00374706 s, 280 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94927 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.88689 s, 21.5 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.250151 s, 67.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00710685 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253445 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0244061 s, 43.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216586 s, 48.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0194215 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0224797 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149803 s, 70.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207312 s, 50.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15625 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.165926 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.319699 s, 3.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.317357 s, 3.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.362704 s, 2.9 MB/s , 0.347431 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.368027 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.357747 s, 2.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00698558 s, 150 MB/s , 0.00707569 s, 148 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107862 s, 97.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00990734 s, 106 MB/s 1048576 bytes (1.0 MB) copied, 0.00860781 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00477278 s, 220 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00915152 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00995872 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106393 s, 98.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108477 s, 96.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120078 s, 87.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115462 s, 90.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117766 s, 89.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117973 s, 88.9 MB/s , 0.0135455 s, 77.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014861 s, 70.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77537 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73362 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.91944 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7292 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.454689 s, 36.9 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132577 s, 79.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191159 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278204 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0221688 s, 47.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0348507 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0334451 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018009 s, 58.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187721 s, 55.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.351319 s, 3.0 MB/s , 0.357841 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.375031 s, 2.8 MB/s , 0.379168 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.367954 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.368383 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.369801 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.378123 s, 2.8 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00638266 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00685462 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00677222 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00941343 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0108018 s, 97.1 MB/s 1048576 bytes (1.0 MB) copied, 0.00786085 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00883922 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0096675 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0099758 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00916776 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00914654 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983506 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111849 s, 93.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010781 s, 97.3 MB/s , 0.0106641 s, 98.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116525 s, 90.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8347 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4295 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.221145 s, 75.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010571 s, 99.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321004 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256819 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.041613 s, 25.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0471452 s, 22.2 MB/s , 0.035281 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0434454 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0467331 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.061903 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0838056 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0993446 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0828526 s, 12.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0949447 s, 11.0 MB/s 1048576 bytes (1.0 MB) copied, 0.107917 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0996401 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0991214 s, 10.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0123476 s, 84.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0176828 s, 59.3 MB/s , 0.0143154 s, 73.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120718 s, 86.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0190288 s, 55.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112061 s, 93.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014319 s, 73.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0179828 s, 58.3 MB/s , 0.0178957 s, 58.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00281192 s, 373 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00258226 s, 406 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123701 s, 84.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00294324 s, 356 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0197748 s, 53.0 MB/s , 0.00259109 s, 405 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00233369 s, 449 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96425 s, 21.1 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74166 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.258718 s, 64.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187013 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0337346 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364215 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281489 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342875 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0442646 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457368 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0461337 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142104 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.16447 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.176371 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.176508 s, 5.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.344558 s, 3.0 MB/s , 0.329057 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.346656 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.343573 s, 3.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154066 s, 68.1 MB/s , 0.0134242 s, 78.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136311 s, 76.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137262 s, 76.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0205515 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167658 s, 62.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168194 s, 62.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00831559 s, 126 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196448 s, 53.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0180315 s, 58.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175473 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166699 s, 62.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183518 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190953 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193887 s, 54.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021124 s, 49.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.84257 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.75297 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.82038 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.74744 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (357s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 04:18:17 (1722241097) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (3s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 04:18:20 (1722241100) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg307-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.97974 s, 263 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg307-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0046 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 04:19:10 (1722241150) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0883916 s, 190 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0105976 s, 98.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0102814 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280938 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0296469 s, 35.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360129 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0387364 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333977 s, 31.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0415243 s, 25.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.044666 s, 23.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0401081 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352188 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371332 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0493773 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.039414 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0408796 s, 25.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3e32800.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3e32800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0445823 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00701448 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00769456 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00156167 s, 671 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00113923 s, 920 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115714 s, 90.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00920637 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00945552 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0089064 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00892836 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00911951 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00305806 s, 343 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00919343 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00979529 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013669 s, 76.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113948 s, 92.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00115215 s, 910 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg307-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93721 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96623 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.192496 s, 87.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165931 s, 63.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0227437 s, 46.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0332991 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315543 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191553 s, 54.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0264377 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268608 s, 39.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0262621 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0684717 s, 15.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0695826 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.16154 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156574 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.268521 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.264865 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.364682 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.361877 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022964 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232647 s, 45.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246394 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229416 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0244956 s, 42.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246417 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237634 s, 44.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282587 s, 37.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259255 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0262644 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270008 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263853 s, 39.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0273877 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0264505 s, 39.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274996 s, 38.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02826 s, 37.1 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg307-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg307-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94275 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96645 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (62s) == sanityn test 77o: Changing rank should not panic ====== 04:20:12 (1722241212) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (3s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 04:20:15 (1722241215) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (26s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 04:20:41 (1722241241) mds.MDS.mdt.nrs_policies=tbf oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg307-client: oleg307-server: ssh exited with exit code 36 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg307-client: oleg307-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (10s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 04:20:51 (1722241251) oleg307-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: Resource temporarily unavailable pdsh@oleg307-client: oleg307-server: ssh exited with exit code 11 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (2s) == sanityn test 79: xattr: intent error ================== 04:20:53 (1722241253) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505431 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-29 04:20:54.000000000 -0400 Modify: 2024-07-29 04:20:54.000000000 -0400 Change: 2024-07-29 04:20:54.000000000 -0400 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 04:21:05 (1722241265) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 04:21:06 (1722241266) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 04:21:07 (1722241267) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (0s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 04:21:07 (1722241267) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 04:21:08 (1722241268) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 04:21:09 (1722241269) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6837 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6837 PASS 82 (1s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 04:21:10 (1722241270) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 04:21:11 (1722241271) fail_loc=0x8000060b PASS 84 (7s) == sanityn test 90: open/create and unlink striped directory ========================================================== 04:21:18 (1722241278) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (1s) == sanityn test 91: chmod and unlink striped directory === 04:21:19 (1722241279) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (0s) == sanityn test 92: create remote directory under orphan directory ========================================================== 04:21:19 (1722241279) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 04:21:20 (1722241280) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2644 0xa54 0 1 2626 0xa42 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2645 0xa55 0 1 2627 0xa43 0 rc1=0 and rc2=0 PASS 93 (8s) == sanityn test 94: signal vs CP callback race =========== 04:21:28 (1722241288) osc.lustre-OST0000-osc-ffff8800aa4c3800.idle_timeout=0 osc.lustre-OST0000-osc-ffff8800b3e32800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800aa4c3800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b3e32800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0358509 s, 229 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 2.99755 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5164: 4574 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0135667 s, 302 kB/s osc.lustre-OST0000-osc-ffff8800aa4c3800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b3e32800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aa4c3800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b3e32800.idle_timeout=debug PASS 94 (9s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 04:21:38 (1722241298) llite.lustre-ffff8800aa4c3800.fast_read=0 llite.lustre-ffff8800b3e32800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00132168 s, 9.3 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800aa4c3800.fast_read=1 llite.lustre-ffff8800b3e32800.fast_read=1 PASS 95b (8s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 04:21:45 (1722241305) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 04:21:46 (1722241306) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0015801 s, 2.6 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (1s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 04:21:47 (1722241307) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0169161 s, 124 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 04:21:49 (1722241309) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0241544 s, 86.8 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (1s) == sanityn test 100e: DoM: read on open and file size ==== 04:21:50 (1722241310) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00137027 s, 8.8 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 04:21:52 (1722241312) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000513876 s, 8.0 MB/s PASS 101a (2s) == sanityn test 101b: Discard DoM data on rename ========= 04:21:54 (1722241314) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000649137 s, 6.3 MB/s PASS 101b (1s) == sanityn test 101c: Discard DoM data on close-unlink === 04:21:55 (1722241315) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000481361 s, 8.5 MB/s PASS 101c (3s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 04:21:58 (1722241318) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1722241319 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1722241321 Finishing test test23 at 1722241322 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 04:22:05 (1722241325) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (1s) == sanityn test 105: Glimpse and lock cancel race ======== 04:22:06 (1722241326) fail_loc=0x416 PASS 105 (47s) == sanityn test 106a: Verify the btime via statx() ======= 04:22:53 (1722241373) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (0s) == sanityn test 106b: Glimpse RPCs test for statx ======== 04:22:54 (1722241374) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) == sanityn test 106c: Verify statx attributes mask ======= 04:22:54 (1722241374) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) == sanityn test 107a: Basic grouplock conflict =========== 04:22:55 (1722241375) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0604443 s, 173 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2652 0xa5c 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6837 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6837 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 04:22:59 (1722241379) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0595352 s, 176 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2631 0xa47 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6837 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (7s) == sanityn test 108a: lseek: parallel updates ============ 04:23:06 (1722241386) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00133312 s, 7.5 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.23865 s, 2.0 MB/s PASS 108a (6s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 04:23:12 (1722241392) 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 7 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 10 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 14 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 17 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 22 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 23 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 25 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 29 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 33 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 34 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 36 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions oleg307-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 46 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 192.168.203.107@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.203.107@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg307-client.virtnet /mnt/lustre (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg307-server' oleg307-server: oleg307-server.virtnet: executing load_modules_local oleg307-server: Loading modules from /home/green/git/lustre-release/lustre oleg307-server: detected 4 online CPUs by sysfs oleg307-server: Force libcfs to create 2 CPU partitions Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre Starting client: oleg307-client.virtnet: -o user_xattr,flock oleg307-server@tcp:/lustre /mnt/lustre2 PASS 109 (387s) == sanityn test 110: do not grant another lock on resend ========================================================== 04:29:39 (1722241779) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2024-07-29 04:29:40.000000000 -0400 Modify: 2024-07-29 04:29:40.000000000 -0400 Change: 2024-07-29 04:29:40.000000000 -0400 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg307-client.virtnet oleg307-client.virtnet PASS 110 (36s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 04:30:15 (1722241815) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 04:30:16 (1722241816) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 04:30:17 (1722241817) lustre-OST0000-osc-ffff880136885800 active. lustre-OST0001-osc-ffff880136885800 active. lustre-OST0000-osc-ffff880136885800 active. lustre-OST0001-osc-ffff880136885800 active. MGC192.168.203.107@tcp active. PASS 113 (1s) cleanup: ====================================================== == sanityn test complete, duration 3917 sec ============== 04:30:19 (1722241819) Stopping clients: oleg307-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg307-client.virtnet /mnt/lustre2 opts: