-----============= acceptance-small: sanityn ============----- Tue Jan 7 11:30:50 EST 2025 excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 Started clients oleg126-client.virtnet: 192.168.201.126@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt) oleg126-client.virtnet: executing check_config_client /mnt/lustre oleg126-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg126-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800aa843000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800aaac9800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aa843000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aaac9800.idle_timeout=debug disable quota as required oleg126-server: oleg126-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101934 s, 103 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6846] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 11:30:59 (1736267459) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (1s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 11:31:00 (1736267460) -rw-r--r-- 1 root root 0 Jan 7 11:31 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (2s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 11:31:02 (1736267462) -rw-r--r-- 1 root root 0 Jan 7 11:31 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (1s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 11:31:03 (1736267463) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (1s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 11:31:04 (1736267464) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (2s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 11:31:06 (1736267466) total 1 -rwxrwxrwx 1 root root 0 Jan 7 11:31 f2a -rwxrwxrwx 1 root root 0 Jan 7 11:31 f2b total 1 -rwxrwxrwx 1 root root 0 Jan 7 11:31 f2a -rwxrwxrwx 1 root root 0 Jan 7 11:31 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.0021776 s, 235 kB/s PASS 2e (1s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 11:31:07 (1736267467) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 11:31:08 (1736267468) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0722455 s, 29.0 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (1s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 11:31:09 (1736267469) PASS 3 (2s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 11:31:11 (1736267471) PASS 4 (2s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 11:31:13 (1736267473) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (2s) == sanityn test 6: remove of open file on other node ============================================================================ 11:31:15 (1736267475) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (1s) == sanityn test 7: remove of open directory on other node ======================================================================= 11:31:16 (1736267476) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (2s) == sanityn test 8: remove of open special file on other node ==================================================================== 11:31:18 (1736267478) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (1s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 11:31:19 (1736267479) PASS 9a (1s) == sanityn test 9b: append to striped sparse file ======== 11:31:21 (1736267481) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160079 s, 65.5 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0127492 s, 0.2 kB/s Data read (expecting 'foo'): foo PASS 9b (1s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 11:31:22 (1736267482) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00126513 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00150841 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00128968 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00133797 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0012553 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00126274 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00128003 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00132109 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00135749 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00133558 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00142405 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00127257 s, 0.8 kB/s PASS 10a (2s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 11:31:24 (1736267484) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00227545 s, 1.4 MB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00634029 s, 646 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000220276 s, 13.9 MB/s PASS 10b (1s) == sanityn test 11: execution of file opened for write should return error ============================================================== 11:31:25 (1736267485) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6846 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (2s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 11:31:27 (1736267487) warning: '-runtime' deprecated, use '-t runtime' instead - link 3324 (time 1736267497.63 total 10.00 last 332.30) - link 6479 (time 1736267507.63 total 20.00 last 315.48) - link 9570 (time 1736267517.63 total 30.00 last 309.09) - link 12415 (time 1736267527.64 total 40.01 last 284.44) - link 15089 (time 1736267537.64 total 50.01 last 267.39) - link 17443 (time 1736267547.64 total 60.01 last 235.31) - link 19736 (time 1736267557.64 total 70.01 last 229.29) - link 22122 (time 1736267567.64 total 80.01 last 238.52) - link 24922 (time 1736267577.65 total 90.01 last 279.96) total: 27998 link in 100.00 seconds: 279.97 ops/second using seed 2529812229 running for 100 seconds - stat 10000 (time 1736267491 ; total 3 ; last 3) - stat 20000 (time 1736267493 ; total 5 ; last 2) - stat 30000 (time 1736267496 ; total 8 ; last 3) - stat 40000 (time 1736267499 ; total 11 ; last 3) - stat 50000 (time 1736267502 ; total 14 ; last 3) - stat 60000 (time 1736267505 ; total 17 ; last 3) - stat 70000 (time 1736267507 ; total 19 ; last 2) - stat 80000 (time 1736267510 ; total 22 ; last 3) - stat 90000 (time 1736267513 ; total 25 ; last 3) - stat 100000 (time 1736267516 ; total 28 ; last 3) - stat 110000 (time 1736267519 ; total 31 ; last 3) - stat 120000 (time 1736267522 ; total 34 ; last 3) - stat 130000 (time 1736267525 ; total 37 ; last 3) - stat 140000 (time 1736267529 ; total 41 ; last 4) - stat 150000 (time 1736267532 ; total 44 ; last 3) - stat 160000 (time 1736267535 ; total 47 ; last 3) - stat 170000 (time 1736267539 ; total 51 ; last 4) - stat 180000 (time 1736267546 ; total 58 ; last 7) - stat 190000 (time 1736267554 ; total 66 ; last 8) - stat 200000 (time 1736267561 ; total 73 ; last 7) - stat 210000 (time 1736267568 ; total 80 ; last 7) - stat 220000 (time 1736267574 ; total 86 ; last 6) - stat 230000 (time 1736267579 ; total 91 ; last 5) - stat 240000 (time 1736267585 ; total 97 ; last 6) total: 244774 stats in 100 seconds: 2447.739990 stats/second - unlinked 0 (time 1736267538 ; total 0 ; last 0) - unlinked 10000 (time 1736267591 ; total 53 ; last 53) - unlinked 20000 (time 1736267611 ; total 73 ; last 20) unlink(/mnt/lustre2/lockdir/lockfile27998) error: No such file or directory total: 27997 unlinks in 89 seconds: 314.573029 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (20439) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (20442) - No such process PASS 12 (141s) == sanityn test 13: test directory page revocation ======= 11:33:48 (1736267628) f13 f13 PASS 13 (1s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 11:33:49 (1736267629) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6846 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4260: 21817 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 11:33:51 (1736267631) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4260: 22444 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (1s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 11:33:52 (1736267632) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 23070 Terminated $DIR1/$tdir/sleep 60 PASS 14b (1s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 11:33:53 (1736267633) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 23698 Terminated $DIR1/$tdir/sleep 60 PASS 14c (2s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 11:33:55 (1736267635) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 24327 Terminated $DIR1/$tdir/sleep 60 PASS 14d (1s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 11:33:56 (1736267636) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7520256 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7520256kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (10s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 11:34:06 (1736267646) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0660168 s, 31.8 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0581285 s, 36.1 MB/s Chance of close/open is 1 in 50 Seed set to 7647 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[1] 1736267647.818228 write 0x83aebe thru 0x841f41 (0x7084 bytes) truncating to largest ever: 0x8eceb2 000003[1] 1736267647.832562 trunc from 0x841f42 to 0x8eceb1 (0xaaf70 bytes) 000005[0] 1736267647.842775 write 0x7a6bc4 thru 0x7ac27e (0x56bb bytes) 000008[1] 1736267647.855980 mapwrite 0x1aed9d thru 0x1bd39f (0xe603 bytes) 000012[1] 1736267647.892007 mapwrite 0x394eab thru 0x39a545 (0x569b bytes) 000013[1] 1736267647.911708 mapwrite 0x60a20d thru 0x617354 (0xd148 bytes) 000014[1] 1736267647.934520 mapwrite 0x621c1c thru 0x62a399 (0x877e bytes) truncating to largest ever: 0x9b906d 000016[0] 1736267647.957834 trunc from 0x8eceb2 to 0x9b906c (0xcc1bb bytes) 000017[1] 1736267647.967925 mapwrite 0x3f76f1 thru 0x3fe811 (0x7121 bytes) 000021[0] 1736267647.997033 mapwrite 0x77e90c thru 0x78102a (0x271f bytes) 000022[0] 1736267648.014960 mapread 0x544e21 thru 0x54a244 (0x5424 bytes) 000023[1] 1736267648.022349 trunc from 0x9b906d to 0x1a40c3 (0x814fa9 bytes) 000025[0] 1736267648.036625 mapwrite 0x8ce7ff thru 0x8dd903 (0xf105 bytes) 000026[1] 1736267648.064679 write 0x5ee558 thru 0x5f449d (0x5f46 bytes) 000027[0] 1736267648.070951 write 0x597a13 thru 0x59c7a7 (0x4d95 bytes) 000031[0] 1736267648.084013 trunc from 0x8dd904 to 0x14482f (0x7990d4 bytes) 000035[1] 1736267648.092654 read 0x00dbe0 thru 0x019b14 (0xbf35 bytes) 000037[1] 1736267648.100886 write 0x7915ef thru 0x79f9af (0xe3c1 bytes) 000038[0] 1736267648.104830 trunc from 0x79f9b0 to 0x477baa (0x327e05 bytes) 000042[1] 1736267648.123294 mapread 0x0fcf5a thru 0x10722d (0xa2d4 bytes) 000043[1] 1736267648.133799 mapread 0x094ff8 thru 0x0a0081 (0xb08a bytes) 000044[1] 1736267648.138438 mapwrite 0x34508d thru 0x3531ad (0xe121 bytes) 000046[1] 1736267648.162419 read 0x28a8b0 thru 0x298c5b (0xe3ac bytes) 000047[1] 1736267648.168439 mapread 0x0a5f26 thru 0x0a98af (0x398a bytes) 000048[1] 1736267648.175850 trunc from 0x477bab to 0x40502f (0x72b7b bytes) 000049[0] 1736267648.181780 mapwrite 0x6448a7 thru 0x64d483 (0x8bdd bytes) 000050[1] 1736267648.207073 mapread 0x5320a8 thru 0x536393 (0x42ec bytes) 000052[0] 1736267648.216076 read 0x4c6f41 thru 0x4d293e (0xb9fe bytes) 000053[1] 1736267648.218075 mapwrite 0x6b6abb thru 0x6beb01 (0x8047 bytes) 000056[1] 1736267648.251869 write 0x54c11c thru 0x551630 (0x5515 bytes) 000057[1] 1736267648.254094 mapwrite 0x3a74a0 thru 0x3adb9e (0x66ff bytes) 000058[0] 1736267648.275080 trunc from 0x6beb02 to 0x3eae6c (0x2d3c95 bytes) 000059[0] 1736267648.285098 trunc from 0x3eae6d to 0x96876e (0x57d902 bytes) 000060[1] 1736267648.293608 mapread 0x84dae0 thru 0x851ca4 (0x41c5 bytes) 000062[0] 1736267648.307809 read 0x7bfff1 thru 0x7c6506 (0x6516 bytes) 000064[0] 1736267648.320448 mapread 0x3622a3 thru 0x366086 (0x3de4 bytes) 000065[0] 1736267648.325453 write 0x3a20c7 thru 0x3b1d91 (0xfccb bytes) 000069[1] 1736267648.336169 write 0x91132f thru 0x91def5 (0xcbc7 bytes) 000070[0] 1736267648.344941 write 0x642634 thru 0x6458a9 (0x3276 bytes) 000072[0] 1736267648.354395 write 0x1e1b12 thru 0x1e384c (0x1d3b bytes) 000073[1] 1736267648.359570 mapwrite 0x873cf4 thru 0x87c46f (0x877c bytes) 000074[1] 1736267648.388630 mapwrite 0x4496be thru 0x44c8c1 (0x3204 bytes) 000076[0] 1736267648.412286 mapread 0x04b8bd thru 0x053390 (0x7ad4 bytes) 000077[0] 1736267648.425534 mapread 0x67f8c9 thru 0x68d19a (0xd8d2 bytes) 000079[1] 1736267648.441302 write 0x63bd42 thru 0x649ece (0xe18d bytes) 000080[0] 1736267648.447807 mapread 0x80d190 thru 0x8149af (0x7820 bytes) 000081[1] 1736267648.464424 mapread 0x69395e thru 0x696b5b (0x31fe bytes) 000082[1] 1736267648.474019 trunc from 0x96876f to 0x23b445 (0x72d329 bytes) 000084[0] 1736267648.491361 read 0x20e05a thru 0x21d830 (0xf7d7 bytes) 000085[1] 1736267648.498420 mapwrite 0x6d65e1 thru 0x6e160f (0xb02f bytes) 000087[0] 1736267648.532675 mapwrite 0x545a33 thru 0x55476b (0xed39 bytes) 000088[1] 1736267648.568835 mapwrite 0x74d3ed thru 0x75319f (0x5db3 bytes) 000090[0] 1736267648.598703 mapwrite 0x1f563f thru 0x2031ce (0xdb90 bytes) 000092[0] 1736267648.652345 trunc from 0x7531a0 to 0x7e9c24 (0x96a85 bytes) 000093[1] 1736267648.657065 mapread 0x34cd12 thru 0x35a0d9 (0xd3c8 bytes) 000097[1] 1736267648.685965 mapwrite 0x4829a1 thru 0x489297 (0x68f7 bytes) 000100[1] 1736267648.718304 read 0x7cee6f thru 0x7db989 (0xcb1b bytes) 000101[0] 1736267648.720898 read 0x465544 thru 0x46ee71 (0x992e bytes) 000102[0] 1736267648.736107 trunc from 0x7e9c25 to 0x4f669e (0x2f3586 bytes) 000103[1] 1736267648.749567 trunc from 0x4f669f to 0x32f66d (0x1c7031 bytes) 000104[0] 1736267648.758327 write 0x162744 thru 0x16fe13 (0xd6d0 bytes) 000105[0] 1736267648.764882 read 0x1cef9a thru 0x1d9fd2 (0xb039 bytes) 000106[0] 1736267648.771774 read 0x1a0b40 thru 0x1a730d (0x67ce bytes) 000109[0] 1736267648.783037 read 0x16c57f thru 0x1771c1 (0xac43 bytes) 000113[1] 1736267648.797678 mapwrite 0x218b41 thru 0x2287f7 (0xfcb7 bytes) 000116[0] 1736267648.840850 write 0x10bc9b thru 0x10d7d7 (0x1b3d bytes) 000118[0] 1736267648.848979 mapread 0x00a442 thru 0x015732 (0xb2f1 bytes) 000119[0] 1736267648.870083 mapread 0x03b672 thru 0x04920d (0xdb9c bytes) 000120[1] 1736267648.885154 mapwrite 0x7e913e thru 0x7f7ea4 (0xed67 bytes) 000125[0] 1736267648.924812 trunc from 0x7f7ea5 to 0x79d835 (0x5a66f bytes) 000126[0] 1736267648.932752 write 0x5ac91b thru 0x5b95eb (0xccd1 bytes) 000127[0] 1736267648.946463 mapwrite 0x8f2482 thru 0x8f4224 (0x1da3 bytes) 000129[0] 1736267648.967509 trunc from 0x8f4225 to 0x897752 (0x5cad2 bytes) 000130[0] 1736267648.970735 read 0x1d0cbc thru 0x1dcbb8 (0xbefd bytes) 000132[0] 1736267648.983124 read 0x07b3a8 thru 0x07b4a8 (0x101 bytes) 000134[1] 1736267648.992318 mapread 0x81ed59 thru 0x8259ee (0x6c96 bytes) 000136[1] 1736267649.007931 mapread 0x2d15c2 thru 0x2d4f30 (0x396f bytes) 000141[0] 1736267649.020012 mapread 0x771fbf thru 0x772d9f (0xde1 bytes) 000142[0] 1736267649.020429 read 0x72018d thru 0x72c97e (0xc7f2 bytes) 000143[0] 1736267649.020753 mapread 0x2f21c2 thru 0x2fecd5 (0xcb14 bytes) 000145[0] 1736267649.034058 read 0x0428af thru 0x04f7ac (0xcefe bytes) 000147[1] 1736267649.051873 mapwrite 0x2e51de thru 0x2e9029 (0x3e4c bytes) 000148[0] 1736267649.078588 read 0x198d96 thru 0x19ec04 (0x5e6f bytes) truncating to largest ever: 0x9ea6e4 000149[0] 1736267649.082026 trunc from 0x897753 to 0x9ea6e3 (0x152f91 bytes) 000152[0] 1736267649.094783 mapread 0x48f4a9 thru 0x490bcd (0x1725 bytes) 000154[1] 1736267649.097930 write 0x12364e thru 0x130814 (0xd1c7 bytes) 000156[0] 1736267649.116127 mapread 0x8e6456 thru 0x8f2a9e (0xc649 bytes) 000157[0] 1736267649.132695 mapread 0x31b528 thru 0x3283c8 (0xcea1 bytes) 000159[1] 1736267649.152127 mapwrite 0x1bb9bf thru 0x1bd821 (0x1e63 bytes) 000160[0] 1736267649.176114 read 0x7ecce9 thru 0x7fb24a (0xe562 bytes) 000161[1] 1736267649.187973 write 0x48af17 thru 0x496c72 (0xbd5c bytes) 000162[0] 1736267649.198276 write 0x09218c thru 0x095148 (0x2fbd bytes) 000163[1] 1736267649.206468 read 0x6c8871 thru 0x6cec3f (0x63cf bytes) 000164[1] 1736267649.213345 mapread 0x7a42dd thru 0x7a99dc (0x5700 bytes) 000166[1] 1736267649.224361 read 0x94df5b thru 0x94f47e (0x1524 bytes) 000168[0] 1736267649.224651 trunc from 0x9ea6e4 to 0x345bd5 (0x6a4b0e bytes) 000169[0] 1736267649.234245 trunc from 0x345bd6 to 0x5651c2 (0x21f5ed bytes) 000171[0] 1736267649.237684 read 0x4fa672 thru 0x505c01 (0xb590 bytes) 000175[0] 1736267649.249883 trunc from 0x5651c3 to 0x5c0c28 (0x5ba66 bytes) 000177[1] 1736267649.259321 read 0x2955d9 thru 0x2a393a (0xe362 bytes) 000180[1] 1736267649.270000 read 0x4b3e44 thru 0x4b47ef (0x9ac bytes) 000182[0] 1736267649.279219 mapwrite 0x9e874d thru 0x9f42fe (0xbbb2 bytes) 000183[1] 1736267649.314502 mapread 0x8b366f thru 0x8b4ff0 (0x1982 bytes) 000185[0] 1736267649.328634 trunc from 0x9f42ff to 0x16f572 (0x884d8c bytes) 000186[0] 1736267649.338994 mapread 0x06eb79 thru 0x07403f (0x54c7 bytes) 000187[0] 1736267649.351109 mapread 0x096839 thru 0x09efbf (0x8787 bytes) 000189[1] 1736267649.364104 mapread 0x0c230d thru 0x0c3113 (0xe07 bytes) 000192[1] 1736267649.382778 mapwrite 0x23e8e0 thru 0x24638d (0x7aae bytes) 000194[0] 1736267649.417691 read 0x121644 thru 0x1218c0 (0x27d bytes) 000196[1] 1736267649.431704 mapwrite 0x2e73a7 thru 0x2ea534 (0x318e bytes) 000205[0] 1736267649.478194 mapread 0x0570b9 thru 0x05e584 (0x74cc bytes) 000207[1] 1736267649.495629 trunc from 0x2ea535 to 0x685cd0 (0x39b79c bytes) 000208[1] 1736267649.506505 write 0x42135c thru 0x423994 (0x2639 bytes) 000210[1] 1736267649.510766 read 0x001664 thru 0x00b9e0 (0xa37d bytes) 000211[0] 1736267649.516515 trunc from 0x685cd1 to 0x19e480 (0x4e7850 bytes) 000213[1] 1736267649.530432 mapread 0x1346ed thru 0x13bf4a (0x785e bytes) 000219[1] 1736267649.550053 mapread 0x0c29f2 thru 0x0cf7ff (0xce0e bytes) 000221[0] 1736267649.557629 read 0x146e92 thru 0x15631e (0xf48d bytes) 000222[0] 1736267649.561971 mapwrite 0x36f775 thru 0x36fdcc (0x658 bytes) 000223[0] 1736267649.592298 write 0x733fa2 thru 0x737e72 (0x3ed1 bytes) 000225[0] 1736267649.599999 mapwrite 0x46e98c thru 0x47914c (0xa7c1 bytes) 000228[0] 1736267649.629531 read 0x50d678 thru 0x51c7a4 (0xf12d bytes) 000229[0] 1736267649.633218 read 0x15d415 thru 0x162ecb (0x5ab7 bytes) 000231[1] 1736267649.644402 mapwrite 0x7ac7e7 thru 0x7b69e3 (0xa1fd bytes) 000232[1] 1736267649.674614 mapwrite 0x8cce09 thru 0x8cf4d9 (0x26d1 bytes) 000233[0] 1736267649.698575 mapwrite 0x8cb20d thru 0x8cc0d5 (0xec9 bytes) 000236[1] 1736267649.726979 trunc from 0x8cf4da to 0x982b1f (0xb3646 bytes) 000240[0] 1736267649.739044 mapwrite 0x20028a thru 0x201e74 (0x1beb bytes) 000241[0] 1736267649.763939 mapwrite 0x54db9e thru 0x54e377 (0x7da bytes) 000242[0] 1736267649.790163 mapread 0x51dd4e thru 0x52a402 (0xc6b5 bytes) 000243[0] 1736267649.803887 trunc from 0x982b20 to 0x22d7e8 (0x755337 bytes) 000244[1] 1736267649.811403 mapread 0x1d89d1 thru 0x1e13d2 (0x8a02 bytes) 000246[1] 1736267649.822741 mapread 0x00703b thru 0x00d860 (0x6826 bytes) 000248[1] 1736267649.831595 read 0x1b7e51 thru 0x1b9578 (0x1728 bytes) 000249[1] 1736267649.835233 mapwrite 0x42c40e thru 0x435fd5 (0x9bc8 bytes) 000251[0] 1736267649.871772 read 0x133f49 thru 0x13cf3c (0x8ff4 bytes) 000253[1] 1736267649.884198 read 0x3b4ef3 thru 0x3bc560 (0x766e bytes) 000254[0] 1736267649.890308 read 0x0a5015 thru 0x0a7674 (0x2660 bytes) 000255[0] 1736267649.903307 mapwrite 0x2fb347 thru 0x3074cd (0xc187 bytes) 000256[0] 1736267649.957841 trunc from 0x435fd6 to 0x8a0aaf (0x46aada bytes) 000258[1] 1736267649.967419 trunc from 0x8a0ab0 to 0x53e654 (0x36245b bytes) 000259[1] 1736267649.976078 mapwrite 0x9815ae thru 0x987c6a (0x66bd bytes) 000260[1] 1736267650.011235 mapwrite 0x333eec thru 0x334adc (0xbf1 bytes) 000262[1] 1736267650.029914 write 0x91c44b thru 0x92070c (0x42c2 bytes) 000263[0] 1736267650.033271 mapread 0x9632ba thru 0x970ceb (0xda32 bytes) 000265[0] 1736267650.059699 trunc from 0x987c6b to 0x76724e (0x220a1c bytes) 000266[1] 1736267650.069655 mapwrite 0x883c49 thru 0x88f3b0 (0xb768 bytes) 000269[0] 1736267650.112284 mapread 0x01c2ab thru 0x01dbed (0x1943 bytes) 000271[0] 1736267650.124860 read 0x6c1289 thru 0x6cc5e9 (0xb361 bytes) 000273[1] 1736267650.129538 read 0x512cbe thru 0x514b35 (0x1e78 bytes) 000274[1] 1736267650.141254 mapread 0x7cb112 thru 0x7d93b8 (0xe2a7 bytes) 000275[1] 1736267650.141612 write 0x7b930a thru 0x7bdbe6 (0x48dd bytes) 000276[1] 1736267650.141962 read 0x668bdd thru 0x66caf1 (0x3f15 bytes) 000277[0] 1736267650.142124 mapread 0x6f071e thru 0x6f21b2 (0x1a95 bytes) 000278[0] 1736267650.146947 mapwrite 0x76d193 thru 0x772cf6 (0x5b64 bytes) 000280[0] 1736267650.175874 write 0x5ac871 thru 0x5b9b06 (0xd296 bytes) 000281[0] 1736267650.178539 write 0x36f066 thru 0x379646 (0xa5e1 bytes) 000286[0] 1736267650.184134 write 0x7ab2e7 thru 0x7adc57 (0x2971 bytes) 000289[1] 1736267650.191659 mapwrite 0x6cad01 thru 0x6da932 (0xfc32 bytes) 000292[0] 1736267650.251206 read 0x853068 thru 0x85cf58 (0x9ef1 bytes) 000293[0] 1736267650.257621 write 0x4b2db1 thru 0x4bb418 (0x8668 bytes) 000295[1] 1736267650.263124 write 0x882b33 thru 0x8898b3 (0x6d81 bytes) 000296[1] 1736267650.272864 mapread 0x8099d9 thru 0x814b55 (0xb17d bytes) 000297[1] 1736267650.288002 trunc from 0x88f3b1 to 0x084294 (0x80b11c bytes) 000298[1] 1736267650.299511 mapwrite 0x20d407 thru 0x21879d (0xb397 bytes) 000299[0] 1736267650.332913 trunc from 0x21879e to 0x9721a6 (0x759a09 bytes) 000300[0] 1736267650.339365 mapread 0x555962 thru 0x55de90 (0x852f bytes) 000301[0] 1736267650.355614 write 0x0006d5 thru 0x007f82 (0x78ae bytes) 000302[0] 1736267650.360412 mapwrite 0x0d5ae2 thru 0x0e47bf (0xecde bytes) 000303[1] 1736267650.389849 mapwrite 0x9c3f0e thru 0x9c7eed (0x3fe0 bytes) 000304[0] 1736267650.416371 mapread 0x49c411 thru 0x4a56b8 (0x92a8 bytes) 000306[0] 1736267650.434084 trunc from 0x9c7eee to 0x1554cc (0x872a21 bytes) 000307[1] 1736267650.443813 mapwrite 0x0e07e5 thru 0x0edfc7 (0xd7e3 bytes) 000308[1] 1736267650.476192 write 0x6b0934 thru 0x6b8daa (0x8477 bytes) 000309[0] 1736267650.481653 write 0x4ad49d thru 0x4b850e (0xb072 bytes) 000312[0] 1736267650.496048 trunc from 0x6b8dab to 0x4c5113 (0x1f3c97 bytes) 000313[1] 1736267650.504013 trunc from 0x4c5114 to 0x01041d (0x4b4cf6 bytes) 000315[0] 1736267650.518483 trunc from 0x01041e to 0x3c7a2b (0x3b760e bytes) 000316[0] 1736267650.524943 mapwrite 0x4b6e0b thru 0x4b8af6 (0x1cec bytes) 000318[0] 1736267650.552039 mapwrite 0x2b8f56 thru 0x2b95dd (0x688 bytes) 000319[0] 1736267650.568348 read 0x0c9216 thru 0x0d4f18 (0xbd03 bytes) 000320[0] 1736267650.570484 mapwrite 0x6097b0 thru 0x60d1a2 (0x39f3 bytes) 000321[1] 1736267650.593728 mapwrite 0x09baa8 thru 0x0a52d4 (0x982d bytes) 000322[1] 1736267650.626956 read 0x0892ac thru 0x098959 (0xf6ae bytes) 000323[1] 1736267650.632856 mapwrite 0x228f8d thru 0x22f468 (0x64dc bytes) 000325[1] 1736267650.658588 read 0x3c8843 thru 0x3d1e39 (0x95f7 bytes) 000328[0] 1736267650.668708 trunc from 0x60d1a3 to 0x909a4d (0x2fc8ab bytes) 000330[0] 1736267650.679300 trunc from 0x909a4e to 0x062884 (0x8a71c9 bytes) 000331[1] 1736267650.682626 mapread 0x05f7ce thru 0x062884 (0x30b7 bytes) 000332[1] 1736267650.692329 mapread 0x021587 thru 0x02ca77 (0xb4f1 bytes) 000334[0] 1736267650.710454 mapread 0x00bec1 thru 0x00dd36 (0x1e76 bytes) 000335[1] 1736267650.717262 write 0x8eff50 thru 0x8fa172 (0xa223 bytes) 000336[1] 1736267650.723888 read 0x808d34 thru 0x817bf4 (0xeec1 bytes) 000337[0] 1736267650.730506 read 0x08ee47 thru 0x093f4a (0x5104 bytes) 000338[0] 1736267650.751809 read 0x18974c thru 0x194e34 (0xb6e9 bytes) 000340[1] 1736267650.758301 mapwrite 0x3a1c6e thru 0x3aa376 (0x8709 bytes) 000342[1] 1736267650.788195 read 0x15bdd8 thru 0x168dd2 (0xcffb bytes) 000345[1] 1736267650.812766 mapread 0x3d5d6b thru 0x3e4676 (0xe90c bytes) 000346[1] 1736267650.813275 mapread 0x167a48 thru 0x176b05 (0xf0be bytes) 000350[1] 1736267650.821937 mapwrite 0x9a9f23 thru 0x9b81eb (0xe2c9 bytes) 000351[0] 1736267650.860331 mapwrite 0x5019ea thru 0x50c9c7 (0xafde bytes) 000352[0] 1736267650.892752 read 0x0b826b thru 0x0bc636 (0x43cc bytes) 000353[0] 1736267650.911807 mapwrite 0x464ab0 thru 0x46f797 (0xace8 bytes) 000354[0] 1736267650.950915 read 0x54b83a thru 0x5594fd (0xdcc4 bytes) 000355[0] 1736267650.954377 trunc from 0x9b81ec to 0x83edb2 (0x179439 bytes) 000357[0] 1736267650.966074 write 0x9a100b thru 0x9ae833 (0xd829 bytes) 000358[0] 1736267650.967291 trunc from 0x9ae834 to 0x399050 (0x6157e3 bytes) 000359[1] 1736267650.975070 trunc from 0x399051 to 0x54295d (0x1a990d bytes) 000360[1] 1736267650.985038 write 0x8ad88d thru 0x8ad928 (0x9c bytes) 000361[1] 1736267650.986062 read 0x443e3e thru 0x44edc9 (0xaf8c bytes) 000362[1] 1736267650.996238 mapread 0x1a0940 thru 0x1a2415 (0x1ad6 bytes) 000363[0] 1736267651.003736 mapwrite 0x611df2 thru 0x61293c (0xb4b bytes) 000365[1] 1736267651.030981 write 0x1caabb thru 0x1d4d03 (0xa249 bytes) 000366[1] 1736267651.036219 mapwrite 0x6040f6 thru 0x610fda (0xcee5 bytes) 000367[1] 1736267651.069016 read 0x8708aa thru 0x87fba6 (0xf2fd bytes) 000369[0] 1736267651.074334 mapwrite 0x2036c9 thru 0x212117 (0xea4f bytes) 000370[1] 1736267651.110889 write 0x31fbf4 thru 0x329f8b (0xa398 bytes) 000372[0] 1736267651.118679 write 0x05d444 thru 0x06a86e (0xd42b bytes) 000373[0] 1736267651.125018 mapwrite 0x31ef2c thru 0x32a528 (0xb5fd bytes) 000374[1] 1736267651.159049 read 0x0ab7ed thru 0x0b1062 (0x5876 bytes) 000375[0] 1736267651.166976 mapwrite 0x232b4a thru 0x23de9f (0xb356 bytes) 000377[0] 1736267651.198065 read 0x57dab8 thru 0x588807 (0xad50 bytes) 000380[0] 1736267651.217524 mapread 0x68b6ea thru 0x69405c (0x8973 bytes) 000381[0] 1736267651.218078 trunc from 0x8ad929 to 0x9488db (0x9afb3 bytes) 000384[0] 1736267651.225633 read 0x5d6991 thru 0x5d69b8 (0x28 bytes) 000385[0] 1736267651.231199 trunc from 0x9488dc to 0x21547a (0x733461 bytes) 000386[1] 1736267651.238355 mapread 0x1030df thru 0x109c9c (0x6bbe bytes) 000387[0] 1736267651.248282 read 0x084e45 thru 0x08b364 (0x6520 bytes) 000388[1] 1736267651.256953 mapread 0x00f3aa thru 0x0141c1 (0x4e18 bytes) 000390[1] 1736267651.266908 trunc from 0x21547b to 0x95da66 (0x7485ec bytes) 000391[0] 1736267651.279307 write 0x087935 thru 0x08900e (0x16da bytes) 000393[1] 1736267651.293693 read 0x8ac512 thru 0x8b1c15 (0x5704 bytes) 000398[0] 1736267651.313754 read 0x721e87 thru 0x72a4dd (0x8657 bytes) 000403[1] 1736267651.325911 mapwrite 0x042513 thru 0x044e63 (0x2951 bytes) 000404[0] 1736267651.347513 read 0x42b923 thru 0x43b692 (0xfd70 bytes) 000407[1] 1736267651.361507 write 0x64619b thru 0x652c1b (0xca81 bytes) 000411[1] 1736267651.376990 mapwrite 0x78fc01 thru 0x79260f (0x2a0f bytes) 000412[0] 1736267651.393989 mapwrite 0x76573d thru 0x767803 (0x20c7 bytes) 000413[0] 1736267651.415177 trunc from 0x95da67 to 0x8e1984 (0x7c0e2 bytes) 000414[0] 1736267651.425689 mapread 0x8a0d69 thru 0x8ab380 (0xa618 bytes) 000415[1] 1736267651.440515 read 0x5806da thru 0x58b086 (0xa9ad bytes) 000418[1] 1736267651.451215 mapread 0x4651fb thru 0x4697c6 (0x45cc bytes) 000419[1] 1736267651.468621 mapwrite 0x4a9cd5 thru 0x4b1fa7 (0x82d3 bytes) 000420[1] 1736267651.497740 read 0x0307df thru 0x037261 (0x6a83 bytes) 000422[1] 1736267651.520060 write 0x0bbf1d thru 0x0c1f12 (0x5ff6 bytes) 000423[1] 1736267651.523696 write 0x01e60b thru 0x02788b (0x9281 bytes) 000425[0] 1736267651.526167 read 0x062810 thru 0x06fac9 (0xd2ba bytes) 000428[1] 1736267651.555935 mapread 0x2427e8 thru 0x249cdf (0x74f8 bytes) 000429[1] 1736267651.564600 trunc from 0x8e1985 to 0x2ec15e (0x5f5826 bytes) 000430[0] 1736267651.579033 mapread 0x037dbd thru 0x03a917 (0x2b5b bytes) 000432[0] 1736267651.590334 write 0x071c28 thru 0x074e78 (0x3251 bytes) 000435[0] 1736267651.602278 mapwrite 0x6474f1 thru 0x64e7a1 (0x72b1 bytes) 000438[1] 1736267651.637400 write 0x2c50c9 thru 0x2cfed2 (0xae0a bytes) 000440[0] 1736267651.647031 mapread 0x2ce5f0 thru 0x2db617 (0xd028 bytes) 000441[0] 1736267651.665317 write 0x16ccd0 thru 0x176d06 (0xa037 bytes) 000444[0] 1736267651.668120 mapread 0x2af751 thru 0x2be86f (0xf11f bytes) 000446[1] 1736267651.681629 mapread 0x5c814f thru 0x5cd604 (0x54b6 bytes) 000453[0] 1736267651.693514 write 0x7a3924 thru 0x7a589b (0x1f78 bytes) 000454[1] 1736267651.698272 trunc from 0x7a589c to 0x1b2547 (0x5f3354 bytes) 000458[0] 1736267651.713883 write 0x6c3583 thru 0x6d2e1e (0xf89c bytes) 000459[1] 1736267651.720535 read 0x0a0065 thru 0x0a8a9f (0x8a3b bytes) 000460[0] 1736267651.742015 mapread 0x360088 thru 0x36e86a (0xe7e3 bytes) 000461[0] 1736267651.763110 write 0x08d636 thru 0x09a915 (0xd2e0 bytes) 000462[1] 1736267651.770388 mapwrite 0x9364a5 thru 0x93681a (0x376 bytes) 000463[0] 1736267651.802638 mapwrite 0x70daf6 thru 0x717223 (0x972e bytes) 000465[1] 1736267651.834209 mapwrite 0x31b82a thru 0x323842 (0x8019 bytes) 000467[0] 1736267651.862774 read 0x54cd2c thru 0x5584de (0xb7b3 bytes) 000468[0] 1736267651.871912 mapwrite 0x9cc64f thru 0x9d0328 (0x3cda bytes) 000469[0] 1736267651.905186 mapread 0x41a201 thru 0x41ffba (0x5dba bytes) 000470[0] 1736267651.912519 mapwrite 0x83476b thru 0x83716e (0x2a04 bytes) 000471[0] 1736267651.926158 mapwrite 0x596080 thru 0x59b261 (0x51e2 bytes) 000472[1] 1736267651.942740 read 0x7f149a thru 0x7fd729 (0xc290 bytes) 000473[1] 1736267651.951190 write 0x891284 thru 0x895d81 (0x4afe bytes) 000474[1] 1736267651.960950 mapwrite 0x82e2a2 thru 0x83bbc0 (0xd91f bytes) 000475[0] 1736267651.974019 trunc from 0x9d0329 to 0x284a45 (0x74b8e3 bytes) 000476[0] 1736267651.987479 write 0x36c418 thru 0x36ca65 (0x64e bytes) 000479[1] 1736267651.996843 mapwrite 0x830d96 thru 0x83ccd3 (0xbf3e bytes) 000480[1] 1736267652.035428 mapread 0x8067bd thru 0x80730f (0xb53 bytes) 000482[0] 1736267652.038734 mapread 0x56304c thru 0x566919 (0x38ce bytes) 000484[1] 1736267652.048706 mapread 0x73ea28 thru 0x74487c (0x5e55 bytes) 000485[0] 1736267652.055095 write 0x6fe5b5 thru 0x70758f (0x8fdb bytes) 000488[0] 1736267652.061697 mapwrite 0x58577c thru 0x586f89 (0x180e bytes) 000489[1] 1736267652.075363 trunc from 0x83ccd4 to 0x80a6fc (0x325d7 bytes) 000490[0] 1736267652.083659 write 0x64c8c5 thru 0x65927b (0xc9b7 bytes) 000492[1] 1736267652.092752 write 0x782c65 thru 0x784284 (0x1620 bytes) 000494[0] 1736267652.101085 mapread 0x00b2c1 thru 0x01745f (0xc19f bytes) 000496[0] 1736267652.118889 mapwrite 0x1035c3 thru 0x104c53 (0x1691 bytes) 000497[1] 1736267652.136150 write 0x8a3b69 thru 0x8b2c07 (0xf09f bytes) 000499[1] 1736267652.146118 read 0x522445 thru 0x525a30 (0x35ec bytes) 000500[1] 1736267652.162306 mapread 0x676e8e thru 0x684262 (0xd3d5 bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 7652 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x5dca2c 000006[1] 1736267652.364358 trunc from 00000000 to 0x5dca2b (0x5dca2c bytes) 000007[1] 1736267652.369349 write_OD 0x9a9000 thru 0x9b1fff (0x9000 bytes) 000009[1] 1736267652.385614 read_OD 0x6da000 thru 0x6e8fff (0xf000 bytes) 000014[0] 1736267652.388457 trunc from 0x9b2000 to 0x016fb6 (0x99b049 bytes) 000018[1] 1736267652.402207 trunc from 0x016fb7 to 0x1190fb (0x102145 bytes) 000019[0] 1736267652.410896 read_OD 0x051000 thru 0x05afff (0xa000 bytes) truncating to largest ever: 0x7097c0 000022[0] 1736267652.418246 trunc from 0x1190fc to 0x7097bf (0x5f06c4 bytes) 000028[0] 1736267652.436511 mapwrite 0x673000 thru 0x680337 (0xd338 bytes) 000029[1] 1736267652.469459 read_OD 0x15a000 thru 0x15dfff (0x4000 bytes) 000030[0] 1736267652.474574 trunc from 0x7097c0 to 0x539e7b (0x1cf944 bytes) 000032[1] 1736267652.483532 write_OD 0x9ba000 thru 0x9c3fff (0xa000 bytes) 000033[0] 1736267652.495334 read_OD 0x4f4000 thru 0x4f9fff (0x6000 bytes) 000036[1] 1736267652.498447 mapread 0x836000 thru 0x842fb2 (0xcfb3 bytes) 000037[1] 1736267652.523165 mapread 0x567000 thru 0x573261 (0xc262 bytes) 000041[1] 1736267652.529240 trunc from 0x9c4000 to 0x23bd70 (0x78828f bytes) 000042[1] 1736267652.541922 read_OD 0x17d000 thru 0x17dfff (0x1000 bytes) 000045[1] 1736267652.546801 read_OD 0x0c3000 thru 0x0ccfff (0xa000 bytes) 000046[1] 1736267652.548795 read_OD 0x234000 thru 0x239fff (0x6000 bytes) 000047[0] 1736267652.550602 read 0x192000 thru 0x1971e2 (0x51e3 bytes) 000050[1] 1736267652.552899 write_OD 0x1f1000 thru 0x1fdfff (0xd000 bytes) 000052[0] 1736267652.565229 trunc from 0x23bd71 to 0x1c9610 (0x72760 bytes) 000057[1] 1736267652.581591 read_OD 0x176000 thru 0x17efff (0x9000 bytes) 000059[1] 1736267652.586419 write_OD 0x9cb000 thru 0x9d0fff (0x6000 bytes) truncating to largest ever: 0x8d2fc4 000060[0] 1736267652.599789 trunc from 0x9d1000 to 0x8d2fc3 (0xfe03c bytes) 000061[1] 1736267652.604929 read_OD 0x30d000 thru 0x317fff (0xb000 bytes) 000062[1] 1736267652.609815 mapwrite 0x8c1000 thru 0x8cc22f (0xb230 bytes) 000063[0] 1736267652.645925 mapread 0x3cc000 thru 0x3d4bd8 (0x8bd9 bytes) 000064[0] 1736267652.659416 trunc from 0x8d2fc4 to 0x2f30fd (0x5dfec6 bytes) 000065[1] 1736267652.664041 mapread 0x215000 thru 0x219b2a (0x4b2b bytes) 000066[0] 1736267652.673322 read 0x1c6000 thru 0x1d2f6c (0xcf6d bytes) 000067[0] 1736267652.680438 trunc from 0x2f30fe to 0x04929d (0x2a9e60 bytes) 000068[0] 1736267652.690159 read 0x017000 thru 0x0236e8 (0xc6e9 bytes) 000069[1] 1736267652.695855 mapread 0x02e000 thru 0x03aede (0xcedf bytes) 000070[1] 1736267652.714349 mapwrite 0x29e000 thru 0x2a8ace (0xaacf bytes) 000071[1] 1736267652.741468 read_OD 0x182000 thru 0x183fff (0x2000 bytes) 000072[0] 1736267652.745517 write 0x1a7000 thru 0x1b69c2 (0xf9c3 bytes) 000073[0] 1736267652.753824 write 0x228000 thru 0x22ff5d (0x7f5e bytes) 000074[0] 1736267652.760683 mapread 0x226000 thru 0x22a617 (0x4618 bytes) 000075[0] 1736267652.765022 read 0x02d000 thru 0x02e5fb (0x15fc bytes) 000076[0] 1736267652.767653 write 0x3d5000 thru 0x3d661d (0x161e bytes) 000078[0] 1736267652.771402 mapwrite 0x72b000 thru 0x737760 (0xc761 bytes) 000079[1] 1736267652.805483 read_OD 0x467000 thru 0x474fff (0xe000 bytes) 000080[0] 1736267652.817240 mapwrite 0x5b9000 thru 0x5c533e (0xc33f bytes) 000081[0] 1736267652.846435 mapwrite 0x6b9000 thru 0x6bbe72 (0x2e73 bytes) 000083[1] 1736267652.863771 trunc from 0x737761 to 0x72c9bf (0xada1 bytes) 000084[0] 1736267652.869292 trunc from 0x72c9c0 to 0x65538e (0xd7631 bytes) 000085[1] 1736267652.877923 mapwrite 0x1cb000 thru 0x1d7ec2 (0xcec3 bytes) 000091[0] 1736267652.919405 mapread 0x3d6000 thru 0x3e510f (0xf110 bytes) 000093[1] 1736267652.937673 read_OD 0x4d7000 thru 0x4d9fff (0x3000 bytes) 000095[0] 1736267652.939924 mapwrite 0x0e6000 thru 0x0ebc2e (0x5c2f bytes) 000096[1] 1736267652.957287 mapwrite 0x1c7000 thru 0x1d45df (0xd5e0 bytes) 000097[1] 1736267652.992491 write_OD 0x8c0000 thru 0x8cbfff (0xc000 bytes) 000099[1] 1736267653.009111 mapread 0x653000 thru 0x65ba2c (0x8a2d bytes) 000100[1] 1736267653.022443 trunc from 0x8cc000 to 0x05e89b (0x86d764 bytes) 000101[1] 1736267653.032899 write_OD 0x968000 thru 0x96afff (0x3000 bytes) 000102[1] 1736267653.046835 mapwrite 0x71a000 thru 0x71e068 (0x4069 bytes) 000103[1] 1736267653.068857 trunc from 0x96b000 to 0x8821f3 (0xe8e0c bytes) 000105[0] 1736267653.077151 read 0x2b3000 thru 0x2bb996 (0x8997 bytes) 000107[1] 1736267653.096743 trunc from 0x8821f4 to 0x2c41ab (0x5be048 bytes) 000108[0] 1736267653.108490 read 0x244000 thru 0x245b0f (0x1b10 bytes) 000109[1] 1736267653.113838 trunc from 0x2c41ac to 0x42e7d9 (0x16a62e bytes) 000116[0] 1736267653.132629 mapread 0x03a000 thru 0x03b9df (0x19e0 bytes) 000117[1] 1736267653.144346 read_OD 0x37a000 thru 0x385fff (0xc000 bytes) 000120[0] 1736267653.146441 write 0x0f5000 thru 0x1042d9 (0xf2da bytes) 000121[0] 1736267653.150724 mapwrite 0x850000 thru 0x85eb0d (0xeb0e bytes) 000122[0] 1736267653.181781 read 0x3e2000 thru 0x3ec4eb (0xa4ec bytes) 000124[1] 1736267653.189756 read_OD 0x422000 thru 0x42ffff (0xe000 bytes) 000125[1] 1736267653.199170 read_OD 0x158000 thru 0x15cfff (0x5000 bytes) 000127[1] 1736267653.201890 write_OD 0x0c7000 thru 0x0cbfff (0x5000 bytes) 000128[1] 1736267653.214942 write_OD 0x2e6000 thru 0x2e9fff (0x4000 bytes) 000129[0] 1736267653.227025 trunc from 0x85eb0e to 0x7718bd (0xed250 bytes) 000131[0] 1736267653.236284 write 0x60b000 thru 0x619c7f (0xec80 bytes) 000134[0] 1736267653.241386 trunc from 0x7718be to 0x85be5e (0xea5a1 bytes) 000136[1] 1736267653.249329 mapread 0x232000 thru 0x233c85 (0x1c86 bytes) 000138[1] 1736267653.280084 read_OD 0x0fd000 thru 0x10afff (0xe000 bytes) 000140[1] 1736267653.282280 mapread 0x12d000 thru 0x12fc49 (0x2c4a bytes) 000142[0] 1736267653.286309 mapwrite 0x749000 thru 0x758b6b (0xfb6c bytes) 000145[0] 1736267653.328359 write 0x180000 thru 0x18ae74 (0xae75 bytes) 000146[1] 1736267653.333058 mapwrite 0x559000 thru 0x559b73 (0xb74 bytes) 000147[0] 1736267653.350787 mapread 0x835000 thru 0x842564 (0xd565 bytes) 000148[1] 1736267653.368320 mapread 0x382000 thru 0x38ad60 (0x8d61 bytes) 000149[0] 1736267653.377652 mapread 0x30c000 thru 0x3128b2 (0x68b3 bytes) 000150[0] 1736267653.384399 write 0x222000 thru 0x231fc9 (0xffca bytes) 000152[1] 1736267653.390232 read_OD 0x718000 thru 0x71cfff (0x5000 bytes) 000153[1] 1736267653.397521 trunc from 0x85be5f to 0x056027 (0x805e37 bytes) 000154[1] 1736267653.403053 mapread 0x02b000 thru 0x03a902 (0xf903 bytes) 000155[0] 1736267653.422668 write 0x934000 thru 0x93ae44 (0x6e45 bytes) 000156[1] 1736267653.429538 mapread 0x53a000 thru 0x540a14 (0x6a15 bytes) 000157[1] 1736267653.451859 write_OD 0x5c6000 thru 0x5ccfff (0x7000 bytes) truncating to largest ever: 0x94c80e 000158[0] 1736267653.466545 trunc from 0x93ae45 to 0x94c80d (0x119c9 bytes) 000159[1] 1736267653.475350 mapread 0x1e0000 thru 0x1ead0c (0xad0d bytes) 000162[0] 1736267653.489782 mapwrite 0x8c0000 thru 0x8cd7be (0xd7bf bytes) 000165[0] 1736267653.519352 read 0x4ff000 thru 0x5008cb (0x18cc bytes) 000169[0] 1736267653.538867 mapwrite 0x54b000 thru 0x554f5b (0x9f5c bytes) 000171[1] 1736267653.556036 write_OD 0x492000 thru 0x49efff (0xd000 bytes) 000172[0] 1736267653.571027 mapwrite 0x389000 thru 0x38a306 (0x1307 bytes) 000175[0] 1736267653.588621 write 0x464000 thru 0x4716b4 (0xd6b5 bytes) 000177[0] 1736267653.593536 mapread 0x264000 thru 0x27245f (0xe460 bytes) 000178[0] 1736267653.607218 mapread 0x8f7000 thru 0x904ce5 (0xdce6 bytes) 000179[1] 1736267653.618554 mapwrite 0x20a000 thru 0x215218 (0xb219 bytes) 000182[1] 1736267653.655191 write_OD 0x857000 thru 0x85efff (0x8000 bytes) 000188[0] 1736267653.668475 mapread 0x5f7000 thru 0x5ff9e5 (0x89e6 bytes) 000190[1] 1736267653.677936 write_OD 0x8ab000 thru 0x8b2fff (0x8000 bytes) 000192[1] 1736267653.690435 read_OD 0x558000 thru 0x560fff (0x9000 bytes) 000194[1] 1736267653.692823 write_OD 0x666000 thru 0x668fff (0x3000 bytes) 000195[0] 1736267653.706140 write 0x1bc000 thru 0x1c0b2f (0x4b30 bytes) 000196[0] 1736267653.711923 mapread 0x756000 thru 0x76482d (0xe82e bytes) 000197[1] 1736267653.728268 mapwrite 0x253000 thru 0x25b189 (0x818a bytes) 000199[0] 1736267653.765943 mapread 0x86b000 thru 0x8763a4 (0xb3a5 bytes) 000200[1] 1736267653.782995 trunc from 0x94c80e to 0x3efbbc (0x55cc51 bytes) 000205[1] 1736267653.796671 mapwrite 0x817000 thru 0x82047a (0x947b bytes) 000208[1] 1736267653.830646 mapwrite 0x6db000 thru 0x6e26b1 (0x76b2 bytes) 000209[1] 1736267653.851574 write_OD 0x2db000 thru 0x2e3fff (0x9000 bytes) 000212[1] 1736267653.865309 mapread 0x390000 thru 0x3982e0 (0x82e1 bytes) 000214[1] 1736267653.875260 write_OD 0x740000 thru 0x744fff (0x5000 bytes) 000217[0] 1736267653.889562 read 0x181000 thru 0x18d7b9 (0xc7ba bytes) 000218[1] 1736267653.908865 trunc from 0x82047b to 0x597288 (0x2891f2 bytes) 000220[1] 1736267653.927749 mapwrite 0x46b000 thru 0x4784d0 (0xd4d1 bytes) 000221[0] 1736267653.964266 read 0x09b000 thru 0x09ccdc (0x1cdd bytes) 000222[0] 1736267653.981022 trunc from 0x597289 to 0x75ffc2 (0x1c8d3a bytes) 000223[0] 1736267653.990303 read 0x1e0000 thru 0x1eab27 (0xab28 bytes) 000225[1] 1736267654.007386 read_OD 0x019000 thru 0x019fff (0x1000 bytes) 000227[0] 1736267654.016866 trunc from 0x75ffc3 to 0x35b587 (0x404a3b bytes) 000228[0] 1736267654.023947 write 0x788000 thru 0x78f3e5 (0x73e6 bytes) 000229[1] 1736267654.028084 mapwrite 0x631000 thru 0x63810f (0x7110 bytes) 000230[1] 1736267654.068822 mapread 0x10e000 thru 0x119638 (0xb639 bytes) 000231[0] 1736267654.072971 write 0x6ad000 thru 0x6b1ca7 (0x4ca8 bytes) 000234[1] 1736267654.085310 write_OD 0x5cb000 thru 0x5d5fff (0xb000 bytes) 000236[1] 1736267654.100191 trunc from 0x78f3e6 to 0x6dcd8c (0xb2659 bytes) 000237[1] 1736267654.105879 trunc from 0x6dcd8d to 0x37ca6f (0x36031d bytes) 000238[1] 1736267654.108843 read_OD 0x257000 thru 0x257fff (0x1000 bytes) 000239[1] 1736267654.111878 read_OD 0x216000 thru 0x21dfff (0x8000 bytes) 000242[1] 1736267654.115816 mapwrite 0x5f2000 thru 0x5f372e (0x172f bytes) 000244[0] 1736267654.155023 read 0x181000 thru 0x18dab5 (0xcab6 bytes) 000249[0] 1736267654.177820 trunc from 0x5f372f to 0x2bcc67 (0x336ac7 bytes) 000254[1] 1736267654.192582 mapwrite 0x0ce000 thru 0x0d5786 (0x7787 bytes) 000256[0] 1736267654.220054 read 0x15a000 thru 0x1638ef (0x98f0 bytes) 000258[1] 1736267654.229551 mapread 0x1fe000 thru 0x2045ef (0x65f0 bytes) 000259[1] 1736267654.236377 mapwrite 0x310000 thru 0x3191c5 (0x91c6 bytes) 000261[0] 1736267654.267258 mapwrite 0x143000 thru 0x152765 (0xf766 bytes) 000262[0] 1736267654.296500 read 0x218000 thru 0x21b733 (0x3734 bytes) 000263[1] 1736267654.300260 mapread 0x1df000 thru 0x1e1db1 (0x2db2 bytes) 000264[0] 1736267654.308133 mapwrite 0x75e000 thru 0x768c58 (0xac59 bytes) 000268[1] 1736267654.346118 read_OD 0x03c000 thru 0x043fff (0x8000 bytes) 000269[0] 1736267654.353611 mapread 0x2a1000 thru 0x2a8cd5 (0x7cd6 bytes) 000271[0] 1736267654.360972 trunc from 0x768c59 to 0x0d8998 (0x6902c0 bytes) 000273[1] 1736267654.371439 write_OD 0x725000 thru 0x725fff (0x1000 bytes) 000274[1] 1736267654.383882 trunc from 0x726000 to 0x5c9b5a (0x15c4a5 bytes) 000275[1] 1736267654.389129 read_OD 0x34a000 thru 0x351fff (0x8000 bytes) 000276[0] 1736267654.393317 mapread 0x4a8000 thru 0x4aa6c1 (0x26c2 bytes) 000277[0] 1736267654.396566 read 0x2e7000 thru 0x2e9267 (0x2268 bytes) 000278[0] 1736267654.405944 mapwrite 0x021000 thru 0x024fa3 (0x3fa4 bytes) 000279[0] 1736267654.422986 mapwrite 0x911000 thru 0x91523b (0x423c bytes) 000283[1] 1736267654.454084 write_OD 0x098000 thru 0x09afff (0x3000 bytes) 000289[0] 1736267654.468142 write 0x951000 thru 0x95fee0 (0xeee1 bytes) 000292[1] 1736267654.473274 trunc from 0x95fee1 to 0x6e1f73 (0x27df6d bytes) 000295[1] 1736267654.484119 mapread 0x4cf000 thru 0x4d6ae6 (0x7ae7 bytes) 000300[0] 1736267654.500371 mapwrite 0x269000 thru 0x27218d (0x918e bytes) 000301[1] 1736267654.528807 read_OD 0x45b000 thru 0x45efff (0x4000 bytes) 000302[1] 1736267654.533298 mapwrite 0x460000 thru 0x46411b (0x411c bytes) 000303[0] 1736267654.558066 mapwrite 0x4a8000 thru 0x4a9994 (0x1995 bytes) 000304[1] 1736267654.577899 write_OD 0x829000 thru 0x82efff (0x6000 bytes) 000307[1] 1736267654.590197 trunc from 0x82f000 to 0x796a39 (0x985c6 bytes) 000308[0] 1736267654.597674 trunc from 0x796a3a to 0x73d9df (0x5905a bytes) 000309[0] 1736267654.602048 write 0x82e000 thru 0x83c837 (0xe838 bytes) 000311[0] 1736267654.609225 mapwrite 0x3cf000 thru 0x3d17fa (0x27fb bytes) 000313[0] 1736267654.624482 mapwrite 0x997000 thru 0x997985 (0x986 bytes) 000315[1] 1736267654.643968 trunc from 0x997986 to 0x5cedba (0x3c8bcb bytes) 000316[0] 1736267654.649023 mapread 0x19f000 thru 0x1a5d13 (0x6d14 bytes) 000317[0] 1736267654.657821 write 0x011000 thru 0x01963e (0x863f bytes) 000319[0] 1736267654.663806 mapwrite 0x6af000 thru 0x6bb7f5 (0xc7f6 bytes) 000320[1] 1736267654.687994 mapread 0x169000 thru 0x16e8a0 (0x58a1 bytes) 000322[0] 1736267654.697775 mapread 0x5ab000 thru 0x5b7102 (0xc103 bytes) 000325[0] 1736267654.708542 mapread 0x0f3000 thru 0x0fe913 (0xb914 bytes) 000329[0] 1736267654.718387 trunc from 0x6bb7f6 to 0x0ec1f6 (0x5cf5ff bytes) 000332[0] 1736267654.729419 trunc from 0x0ec1f7 to 0x6c1ea9 (0x5d5cb3 bytes) 000334[1] 1736267654.734330 mapwrite 0x2b2000 thru 0x2be534 (0xc535 bytes) 000336[1] 1736267654.763265 trunc from 0x6c1eaa to 0x5e90f4 (0xd8db5 bytes) 000337[0] 1736267654.767363 write 0x941000 thru 0x947c8e (0x6c8f bytes) 000338[0] 1736267654.770786 mapwrite 0x179000 thru 0x183ffe (0xafff bytes) 000339[0] 1736267654.795609 read 0x7a4000 thru 0x7a85ee (0x45ef bytes) 000340[1] 1736267654.806303 mapread 0x7bf000 thru 0x7c36b0 (0x46b1 bytes) 000341[1] 1736267654.820024 trunc from 0x947c8f to 0x0ecf9d (0x85acf1 bytes) 000342[0] 1736267654.827944 read 0x0ad000 thru 0x0bc268 (0xf269 bytes) 000343[0] 1736267654.833801 write 0x347000 thru 0x35592a (0xe92b bytes) 000345[1] 1736267654.837551 mapwrite 0x875000 thru 0x88428e (0xf28f bytes) 000347[0] 1736267654.878873 trunc from 0x88428f to 0x2628b5 (0x6219d9 bytes) 000351[1] 1736267654.891205 mapread 0x209000 thru 0x216df5 (0xddf6 bytes) 000352[0] 1736267654.907052 mapwrite 0x062000 thru 0x069730 (0x7731 bytes) 000353[1] 1736267654.932860 read_OD 0x1dd000 thru 0x1eafff (0xe000 bytes) 000354[1] 1736267654.938903 write_OD 0x0fa000 thru 0x0fefff (0x5000 bytes) 000356[1] 1736267654.951066 mapread 0x078000 thru 0x07b006 (0x3007 bytes) 000358[0] 1736267654.957221 write 0x1a8000 thru 0x1ab103 (0x3104 bytes) 000359[0] 1736267654.960667 write 0x59e000 thru 0x5a068e (0x268f bytes) 000360[0] 1736267654.964409 write 0x816000 thru 0x816a5f (0xa60 bytes) 000362[0] 1736267654.970550 mapread 0x667000 thru 0x66dde1 (0x6de2 bytes) 000365[1] 1736267654.988683 mapwrite 0x86e000 thru 0x86fffa (0x1ffb bytes) 000367[0] 1736267655.031810 trunc from 0x86fffb to 0x750f40 (0x11f0ba bytes) 000368[0] 1736267655.041212 mapread 0x36a000 thru 0x37443c (0xa43d bytes) 000369[0] 1736267655.051196 write 0x45e000 thru 0x460132 (0x2133 bytes) 000370[1] 1736267655.052208 write_OD 0x20a000 thru 0x213fff (0xa000 bytes) 000372[1] 1736267655.066188 mapread 0x469000 thru 0x47124e (0x824f bytes) 000373[0] 1736267655.074068 mapread 0x417000 thru 0x420f50 (0x9f51 bytes) 000374[1] 1736267655.084108 trunc from 0x750f41 to 0x160bc5 (0x5f037b bytes) 000379[0] 1736267655.097679 write 0x8b9000 thru 0x8b94d5 (0x4d6 bytes) 000382[1] 1736267655.106210 mapwrite 0x245000 thru 0x252482 (0xd483 bytes) 000385[0] 1736267655.137871 write 0x238000 thru 0x240d91 (0x8d92 bytes) 000386[1] 1736267655.143870 mapread 0x88b000 thru 0x895924 (0xa925 bytes) 000388[0] 1736267655.159703 mapread 0x104000 thru 0x10c214 (0x8215 bytes) 000390[1] 1736267655.167353 trunc from 0x8b94d6 to 0x86ece2 (0x4a7f3 bytes) 000391[0] 1736267655.172693 read 0x541000 thru 0x54b287 (0xa288 bytes) 000393[0] 1736267655.185346 mapwrite 0x731000 thru 0x73333d (0x233e bytes) 000394[1] 1736267655.217625 mapread 0x3a5000 thru 0x3a6ac5 (0x1ac6 bytes) 000395[0] 1736267655.223169 mapwrite 0x262000 thru 0x267570 (0x5571 bytes) 000396[0] 1736267655.243394 mapwrite 0x657000 thru 0x65cd9c (0x5d9d bytes) 000397[0] 1736267655.263268 mapwrite 0x5a9000 thru 0x5b8920 (0xf921 bytes) 000398[0] 1736267655.294429 mapwrite 0x947000 thru 0x9492b6 (0x22b7 bytes) 000401[1] 1736267655.319759 trunc from 0x9492b7 to 0x797f38 (0x1b137e bytes) 000403[1] 1736267655.326644 mapread 0x330000 thru 0x336574 (0x6575 bytes) 000405[0] 1736267655.337407 read 0x6dc000 thru 0x6e5f94 (0x9f95 bytes) 000407[1] 1736267655.343543 mapwrite 0x5ac000 thru 0x5b919e (0xd19f bytes) 000412[1] 1736267655.376671 write_OD 0x51d000 thru 0x51dfff (0x1000 bytes) 000413[1] 1736267655.388129 mapwrite 0x24f000 thru 0x255ee6 (0x6ee7 bytes) 000414[1] 1736267655.409107 read_OD 0x77b000 thru 0x784fff (0xa000 bytes) 000417[0] 1736267655.413213 read 0x487000 thru 0x489c55 (0x2c56 bytes) 000418[1] 1736267655.421697 mapwrite 0x095000 thru 0x0967f5 (0x17f6 bytes) 000419[0] 1736267655.438759 read 0x0de000 thru 0x0e7323 (0x9324 bytes) 000420[1] 1736267655.460111 mapread 0x766000 thru 0x775e2f (0xfe30 bytes) 000421[0] 1736267655.474368 read 0x554000 thru 0x55470b (0x70c bytes) 000422[0] 1736267655.474542 mapread 0x46c000 thru 0x478ec5 (0xcec6 bytes) 000423[0] 1736267655.474766 mapwrite 0x8c7000 thru 0x8c836f (0x1370 bytes) 000425[0] 1736267655.503936 read 0x098000 thru 0x09f505 (0x7506 bytes) 000426[1] 1736267655.528242 trunc from 0x8c8370 to 0x8d1f67 (0x9bf8 bytes) 000427[0] 1736267655.540468 mapwrite 0x2f3000 thru 0x3004f2 (0xd4f3 bytes) 000428[1] 1736267655.573826 mapread 0x113000 thru 0x114d1e (0x1d1f bytes) 000430[1] 1736267655.585643 trunc from 0x8d1f68 to 0x09dcb6 (0x8342b1 bytes) 000431[1] 1736267655.592770 mapwrite 0x3d0000 thru 0x3d2b52 (0x2b53 bytes) 000432[0] 1736267655.609863 mapread 0x346000 thru 0x34df4e (0x7f4f bytes) 000433[1] 1736267655.621256 mapwrite 0x3e2000 thru 0x3e8e5f (0x6e60 bytes) 000435[0] 1736267655.644345 write 0x394000 thru 0x39d051 (0x9052 bytes) 000438[1] 1736267655.651434 write_OD 0x673000 thru 0x680fff (0xe000 bytes) 000440[1] 1736267655.666905 mapwrite 0x6b1000 thru 0x6b296e (0x196f bytes) 000441[0] 1736267655.685634 trunc from 0x6b296f to 0x90833d (0x2559cf bytes) 000446[0] 1736267655.701661 mapread 0x600000 thru 0x60b0ea (0xb0eb bytes) 000447[0] 1736267655.716150 mapwrite 0x647000 thru 0x651540 (0xa541 bytes) 000448[0] 1736267655.741171 mapread 0x342000 thru 0x34c4e8 (0xa4e9 bytes) 000449[1] 1736267655.751657 trunc from 0x90833e to 0x04a076 (0x8be2c7 bytes) 000451[0] 1736267655.762338 write 0x38c000 thru 0x3975da (0xb5db bytes) 000459[0] 1736267655.771977 write 0x4fa000 thru 0x4fed83 (0x4d84 bytes) 000460[0] 1736267655.775857 write 0x746000 thru 0x7548a2 (0xe8a3 bytes) 000466[1] 1736267655.787458 write_OD 0x97d000 thru 0x986fff (0xa000 bytes) 000468[0] 1736267655.803021 mapwrite 0x411000 thru 0x41d9de (0xc9df bytes) 000469[0] 1736267655.832137 mapread 0x832000 thru 0x834aab (0x2aac bytes) 000472[1] 1736267655.844521 write_OD 0x860000 thru 0x86efff (0xf000 bytes) truncating to largest ever: 0x95622b 000474[1] 1736267655.863192 trunc from 0x987000 to 0x95622a (0x30dd5 bytes) 000475[0] 1736267655.868980 mapread 0x87a000 thru 0x87a290 (0x291 bytes) 000479[1] 1736267655.876294 read_OD 0x028000 thru 0x030fff (0x9000 bytes) 000480[1] 1736267655.878475 mapwrite 0x3c3000 thru 0x3cc29c (0x929d bytes) 000482[0] 1736267655.925127 trunc from 0x95622b to 0x758cee (0x1fd53c bytes) 000483[0] 1736267655.936927 read 0x6dd000 thru 0x6dd378 (0x379 bytes) 000485[0] 1736267655.941596 mapwrite 0x119000 thru 0x124360 (0xb361 bytes) 000486[1] 1736267655.960450 trunc from 0x758cef to 0x680d09 (0xd7fe5 bytes) 000487[1] 1736267655.966456 mapwrite 0x85f000 thru 0x85f97a (0x97b bytes) 000495[1] 1736267655.993148 read_OD 0x690000 thru 0x695fff (0x6000 bytes) 000496[1] 1736267655.996828 mapread 0x5b0000 thru 0x5bbe74 (0xbe75 bytes) 000498[1] 1736267656.014635 mapwrite 0x4f2000 thru 0x4f2737 (0x738 bytes) 000499[1] 1736267656.028846 mapread 0x6b2000 thru 0x6bed3c (0xcd3d bytes) 000500[1] 1736267656.040161 write_OD 0x47a000 thru 0x47cfff (0x3000 bytes) All operations completed A-OK! PASS 16a (10s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 11:34:17 (1736267657) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0637847 s, 32.9 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0499298 s, 42.0 MB/s Chance of close/open is 1 in 50 Seed set to 7657 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read 000003[1] 1736267657.685391 write 0x0d6687 thru 0x0d6b84 (0x4fe bytes) 000006[0] 1736267657.690036 mapwrite 0x783022 thru 0x783ed0 (0xeaf bytes) 000007[1] 1736267657.718794 write 0x3a9025 thru 0x3a9181 (0x15d bytes) 000008[0] 1736267657.725473 mapread 0x496265 thru 0x496a4c (0x7e8 bytes) 000009[0] 1736267657.730973 read 0x361f0f thru 0x36370a (0x17fc bytes) 000012[1] 1736267657.738677 mapread 0x47d137 thru 0x47d5a6 (0x470 bytes) truncating to largest ever: 0x3d66a2 000014[0] 1736267657.744701 trunc from 0x783ed1 to 0x3d66a1 (0x3ad82f bytes) 000015[0] 1736267657.752294 mapwrite 0x129d84 thru 0x12b1aa (0x1427 bytes) 000016[0] 1736267657.766931 read 0x081870 thru 0x081e1c (0x5ad bytes) 000019[0] 1736267657.779092 read 0x29af91 thru 0x29c1ff (0x126f bytes) 000020[0] 1736267657.783077 write 0x5cac5f thru 0x5cc230 (0x15d2 bytes) 000021[0] 1736267657.783779 read 0x542b2e thru 0x5433ed (0x8c0 bytes) 000024[0] 1736267657.790462 mapwrite 0x52d7f5 thru 0x52e2e6 (0xaf2 bytes) 000025[0] 1736267657.806187 write 0x6d0353 thru 0x6d19d4 (0x1682 bytes) 000027[1] 1736267657.812023 write 0x6d3c64 thru 0x6d55a0 (0x193d bytes) truncating to largest ever: 0x85c936 000031[0] 1736267657.827461 trunc from 0x6d55a1 to 0x85c935 (0x187395 bytes) 000033[0] 1736267657.834900 trunc from 0x85c936 to 0x83d011 (0x1f924 bytes) 000034[0] 1736267657.840375 trunc from 0x83d012 to 0x4d0084 (0x36cf8d bytes) 000035[0] 1736267657.847404 read 0x30cc76 thru 0x30d380 (0x70b bytes) 000036[1] 1736267657.852794 mapwrite 0x7d29d5 thru 0x7d3ea9 (0x14d5 bytes) 000037[1] 1736267657.877275 write 0x0cbd9e thru 0x0cc2c8 (0x52b bytes) 000038[0] 1736267657.882442 mapwrite 0x2e5593 thru 0x2e6729 (0x1197 bytes) 000040[1] 1736267657.908229 mapwrite 0x83543c thru 0x836d95 (0x195a bytes) 000041[0] 1736267657.926130 mapwrite 0x2749fb thru 0x27557f (0xb85 bytes) 000042[0] 1736267657.944882 trunc from 0x836d96 to 0x1a3fce (0x692dc7 bytes) 000044[1] 1736267657.954955 mapread 0x09436d thru 0x0947f2 (0x486 bytes) 000045[0] 1736267657.960469 trunc from 0x1a3fcf to 0x402fba (0x25efec bytes) 000046[0] 1736267657.968811 write 0x9972fe thru 0x997a26 (0x729 bytes) 000047[1] 1736267657.971110 trunc from 0x997a27 to 0x28ba4e (0x70bfd8 bytes) 000049[0] 1736267657.983407 write 0x297897 thru 0x298383 (0xaed bytes) 000051[0] 1736267657.991172 trunc from 0x298384 to 0x23c8fc (0x5ba87 bytes) 000052[0] 1736267657.996532 read 0x103f4b thru 0x105a55 (0x1b0b bytes) 000054[0] 1736267658.009507 read 0x05ae80 thru 0x05b96e (0xaef bytes) 000055[0] 1736267658.015742 mapread 0x04f79d thru 0x050c91 (0x14f5 bytes) 000057[1] 1736267658.018098 write 0x9966d2 thru 0x9986b5 (0x1fe4 bytes) 000059[0] 1736267658.028017 mapwrite 0x9500e7 thru 0x950873 (0x78d bytes) 000061[0] 1736267658.069669 mapread 0x86969b thru 0x86a2c3 (0xc29 bytes) 000063[1] 1736267658.070257 mapread 0x43b4f6 thru 0x43cdc2 (0x18cd bytes) 000064[0] 1736267658.084263 mapread 0x521359 thru 0x5213cd (0x75 bytes) 000065[1] 1736267658.085401 mapwrite 0x4b1b6c thru 0x4b31a1 (0x1636 bytes) 000066[0] 1736267658.113402 mapread 0x27ea78 thru 0x27ff97 (0x1520 bytes) 000068[0] 1736267658.124654 read 0x07cae1 thru 0x07ead9 (0x1ff9 bytes) 000069[1] 1736267658.149021 read 0x227cb5 thru 0x228dc1 (0x110d bytes) 000071[0] 1736267658.150287 mapwrite 0x89cbb9 thru 0x89ded2 (0x131a bytes) 000077[0] 1736267658.192810 trunc from 0x9986b6 to 0x0fd1b5 (0x89b500 bytes) 000079[1] 1736267658.203450 mapread 0x04274f thru 0x043da6 (0x1658 bytes) 000080[0] 1736267658.209541 read 0x006b35 thru 0x007eac (0x1378 bytes) 000082[0] 1736267658.223810 mapwrite 0x731c26 thru 0x7336aa (0x1a85 bytes) 000087[1] 1736267658.272591 mapwrite 0x0c4099 thru 0x0c5d24 (0x1c8c bytes) 000089[0] 1736267658.301394 read 0x56fc8a thru 0x570fc8 (0x133f bytes) 000090[0] 1736267658.304548 write 0x60476e thru 0x605bde (0x1471 bytes) 000092[0] 1736267658.309489 mapread 0x0eee74 thru 0x0f0d40 (0x1ecd bytes) truncating to largest ever: 0x9d333d 000095[1] 1736267658.321278 trunc from 0x7336ab to 0x9d333c (0x29fc92 bytes) 000098[0] 1736267658.337652 trunc from 0x9d333d to 0x8856c6 (0x14dc76 bytes) 000100[0] 1736267658.349109 mapread 0x275276 thru 0x27616a (0xef5 bytes) 000101[1] 1736267658.354656 mapwrite 0x13dafe thru 0x13db4d (0x50 bytes) 000107[0] 1736267658.388854 trunc from 0x8856c7 to 0x3c93e9 (0x4bc2dd bytes) 000108[1] 1736267658.394535 write 0x018612 thru 0x0192c8 (0xcb7 bytes) 000110[1] 1736267658.404427 read 0x2878ec thru 0x288b2f (0x1244 bytes) 000111[0] 1736267658.411998 mapread 0x0672d5 thru 0x0686a5 (0x13d1 bytes) 000113[0] 1736267658.418615 mapread 0x177898 thru 0x178b71 (0x12da bytes) 000114[1] 1736267658.422960 mapread 0x3233eb thru 0x323a65 (0x67b bytes) 000116[1] 1736267658.433387 write 0x9f62d7 thru 0x9f7bcd (0x18f7 bytes) 000117[1] 1736267658.437646 write 0x0681af thru 0x0690c8 (0xf1a bytes) 000121[1] 1736267658.462688 trunc from 0x9f7bce to 0x5aa742 (0x44d48b bytes) 000124[1] 1736267658.475391 write 0x516354 thru 0x5180ac (0x1d59 bytes) 000125[1] 1736267658.478346 mapread 0x2048c0 thru 0x20580b (0xf4c bytes) 000126[0] 1736267658.483348 mapwrite 0x056cbb thru 0x0570a4 (0x3ea bytes) 000129[1] 1736267658.510746 mapread 0x48301a thru 0x483cb3 (0xc9a bytes) 000130[1] 1736267658.515141 write 0x7f559f thru 0x7f64ce (0xf30 bytes) 000131[0] 1736267658.518985 trunc from 0x7f64cf to 0x0f003c (0x706492 bytes) 000132[1] 1736267658.530690 mapwrite 0x74e8d3 thru 0x750269 (0x1997 bytes) 000134[0] 1736267658.559008 trunc from 0x75026a to 0x03395e (0x71c90b bytes) 000135[0] 1736267658.566399 mapwrite 0x3f570c thru 0x3f671d (0x1012 bytes) 000136[1] 1736267658.583844 read 0x2edfc2 thru 0x2efd43 (0x1d82 bytes) 000137[0] 1736267658.588339 write 0x7bf226 thru 0x7bfd83 (0xb5e bytes) 000138[0] 1736267658.589376 trunc from 0x7bfd84 to 0x574681 (0x24b702 bytes) 000145[1] 1736267658.601067 write 0x1f5bc3 thru 0x1f6d1c (0x115a bytes) 000147[0] 1736267658.616537 write 0x985156 thru 0x986474 (0x131f bytes) 000148[0] 1736267658.623778 mapwrite 0x15261b thru 0x15414a (0x1b30 bytes) 000151[1] 1736267658.643254 write 0x3e10e0 thru 0x3e1b29 (0xa4a bytes) 000152[1] 1736267658.648569 mapread 0x634f82 thru 0x636a75 (0x1af4 bytes) 000153[0] 1736267658.669233 write 0x21fdcf thru 0x221446 (0x1678 bytes) 000154[0] 1736267658.677629 mapwrite 0x45d025 thru 0x45d33b (0x317 bytes) 000156[0] 1736267658.696411 write 0x6d4aaf thru 0x6d6926 (0x1e78 bytes) truncating to largest ever: 0x9fed6b 000157[0] 1736267658.699242 trunc from 0x986475 to 0x9fed6a (0x788f6 bytes) 000158[1] 1736267658.709967 mapwrite 0x3ab130 thru 0x3abf59 (0xe2a bytes) 000159[0] 1736267658.727735 mapwrite 0x4c6787 thru 0x4c727a (0xaf4 bytes) 000160[1] 1736267658.745772 write 0x88ed1a thru 0x88f509 (0x7f0 bytes) 000162[0] 1736267658.753153 mapwrite 0x3eb578 thru 0x3ed15d (0x1be6 bytes) 000163[1] 1736267658.774100 mapread 0x04964c thru 0x04b3af (0x1d64 bytes) 000167[0] 1736267658.789241 mapread 0x34f443 thru 0x3502cf (0xe8d bytes) 000168[1] 1736267658.794619 mapread 0x671987 thru 0x6730e6 (0x1760 bytes) 000169[1] 1736267658.801005 mapwrite 0x5b3dd1 thru 0x5b5028 (0x1258 bytes) 000171[1] 1736267658.818803 read 0x407b3c thru 0x4089c3 (0xe88 bytes) 000173[0] 1736267658.842845 trunc from 0x9fed6b to 0x6ae0ed (0x350c7d bytes) 000174[0] 1736267658.856364 trunc from 0x6ae0ee to 0x02e242 (0x67feab bytes) 000175[0] 1736267658.859495 mapwrite 0x00a789 thru 0x00bfa4 (0x181c bytes) 000178[1] 1736267658.879124 read 0x02d188 thru 0x02e242 (0x10bb bytes) 000179[0] 1736267658.882673 mapwrite 0x39942a thru 0x39ad53 (0x192a bytes) 000182[1] 1736267658.904899 write 0x11b75d thru 0x11b7f2 (0x96 bytes) 000183[0] 1736267658.910684 trunc from 0x39ad54 to 0x042954 (0x3583ff bytes) 000185[1] 1736267658.916881 trunc from 0x042955 to 0x852b8c (0x810238 bytes) 000186[1] 1736267658.924999 write 0x149ad3 thru 0x14b2a2 (0x17d0 bytes) 000188[0] 1736267658.949439 trunc from 0x852b8d to 0x213d34 (0x63ee58 bytes) 000189[1] 1736267658.965110 trunc from 0x213d35 to 0x99e839 (0x78ab05 bytes) 000191[0] 1736267658.977566 mapwrite 0x5971df thru 0x597297 (0xb9 bytes) 000192[1] 1736267658.994926 mapread 0x0e8167 thru 0x0e9ecc (0x1d66 bytes) 000195[0] 1736267659.011174 mapwrite 0x75223e thru 0x752a54 (0x817 bytes) 000200[0] 1736267659.034945 read 0x4958c7 thru 0x496338 (0xa72 bytes) 000201[0] 1736267659.053977 write 0x8db47a thru 0x8dcb86 (0x170d bytes) 000203[1] 1736267659.061788 write 0x3cc583 thru 0x3cd1cc (0xc4a bytes) 000204[0] 1736267659.075507 read 0x2014f0 thru 0x201ed2 (0x9e3 bytes) 000206[1] 1736267659.088973 write 0x83425e thru 0x835574 (0x1317 bytes) 000207[0] 1736267659.097946 trunc from 0x99e83a to 0x1ed1ca (0x7b166f bytes) 000208[1] 1736267659.106416 mapwrite 0x4a3409 thru 0x4a3810 (0x408 bytes) 000210[1] 1736267659.125775 read 0x0d89d7 thru 0x0da292 (0x18bc bytes) 000211[1] 1736267659.139031 write 0x325dd6 thru 0x3261f7 (0x422 bytes) 000212[0] 1736267659.143983 mapread 0x1d694d thru 0x1d79d1 (0x1085 bytes) 000213[1] 1736267659.159365 trunc from 0x4a3811 to 0x605251 (0x161a41 bytes) 000216[1] 1736267659.176118 trunc from 0x605252 to 0x13f848 (0x4c5a09 bytes) 000218[0] 1736267659.186559 write 0x0faefb thru 0x0fb1eb (0x2f1 bytes) 000219[1] 1736267659.193408 mapwrite 0x68412d thru 0x684aad (0x981 bytes) 000221[0] 1736267659.221988 read 0x2db627 thru 0x2dc54b (0xf25 bytes) 000222[1] 1736267659.228152 mapwrite 0x25b1b4 thru 0x25caca (0x1917 bytes) 000223[1] 1736267659.246903 mapwrite 0x85698e thru 0x857aac (0x111f bytes) 000224[0] 1736267659.273542 mapread 0x554f8c thru 0x5550b8 (0x12d bytes) 000225[1] 1736267659.289958 read 0x49f49d thru 0x4a124d (0x1db1 bytes) 000227[1] 1736267659.300111 read 0x25b6c7 thru 0x25ba72 (0x3ac bytes) 000228[1] 1736267659.303379 write 0x1371ab thru 0x13854f (0x13a5 bytes) 000230[1] 1736267659.325298 trunc from 0x857aad to 0x1e6424 (0x671688 bytes) 000231[0] 1736267659.333238 mapread 0x155d12 thru 0x15608c (0x37b bytes) 000234[1] 1736267659.343828 trunc from 0x1e6425 to 0x60774d (0x421329 bytes) 000235[1] 1736267659.352522 read 0x5f71b0 thru 0x5f888c (0x16dd bytes) 000236[1] 1736267659.355867 read 0x4a2178 thru 0x4a36b5 (0x153e bytes) 000239[1] 1736267659.362840 mapread 0x1aa8af thru 0x1abba4 (0x12f6 bytes) 000242[1] 1736267659.372122 mapread 0x07b7d6 thru 0x07c35c (0xb87 bytes) 000244[1] 1736267659.379125 mapread 0x57b3a9 thru 0x57cd23 (0x197b bytes) 000247[0] 1736267659.383176 mapwrite 0x2231e7 thru 0x224c17 (0x1a31 bytes) 000248[0] 1736267659.410149 trunc from 0x60774e to 0x677a4b (0x702fe bytes) 000251[1] 1736267659.419406 read 0x3ae575 thru 0x3aee7e (0x90a bytes) 000252[0] 1736267659.427591 mapread 0x4d7faf thru 0x4d8bbf (0xc11 bytes) 000254[0] 1736267659.431375 trunc from 0x677a4c to 0x52b6de (0x14c36d bytes) 000255[1] 1736267659.439591 write 0x65c323 thru 0x65de84 (0x1b62 bytes) 000256[0] 1736267659.445401 mapwrite 0x07b88d thru 0x07d76b (0x1edf bytes) 000258[1] 1736267659.477041 read 0x37f928 thru 0x38037a (0xa53 bytes) 000263[1] 1736267659.492662 trunc from 0x65de85 to 0x608f82 (0x54f02 bytes) 000264[1] 1736267659.500223 write 0x5ce3b4 thru 0x5ce861 (0x4ae bytes) 000265[0] 1736267659.511733 write 0x8e68fe thru 0x8e8452 (0x1b55 bytes) 000267[1] 1736267659.522345 read 0x0f5d62 thru 0x0f673c (0x9db bytes) 000270[0] 1736267659.557150 mapwrite 0x7a11bb thru 0x7a1280 (0xc6 bytes) 000273[1] 1736267659.588126 trunc from 0x8e8453 to 0x8ee977 (0x6525 bytes) 000275[0] 1736267659.605705 trunc from 0x8ee978 to 0x50f474 (0x3df503 bytes) 000277[1] 1736267659.616757 mapwrite 0x819644 thru 0x81a95d (0x131a bytes) 000279[1] 1736267659.641136 read 0x2d49d5 thru 0x2d4b42 (0x16e bytes) 000280[1] 1736267659.657891 mapread 0x1638cf thru 0x163a82 (0x1b4 bytes) 000281[1] 1736267659.661739 trunc from 0x81a95e to 0x2d7659 (0x543304 bytes) 000283[1] 1736267659.673648 mapread 0x18e797 thru 0x18ed95 (0x5ff bytes) 000284[1] 1736267659.674869 trunc from 0x2d765a to 0x947482 (0x66fe29 bytes) 000286[0] 1736267659.683536 write 0x8607af thru 0x86213e (0x1990 bytes) 000287[0] 1736267659.691002 trunc from 0x947483 to 0x64fd42 (0x2f7740 bytes) 000289[1] 1736267659.700693 read 0x4b5449 thru 0x4b6162 (0xd1a bytes) 000290[0] 1736267659.706788 read 0x49aa30 thru 0x49ae5c (0x42d bytes) 000291[1] 1736267659.717407 mapwrite 0x01b860 thru 0x01ca44 (0x11e5 bytes) 000292[0] 1736267659.736419 mapread 0x26a1a3 thru 0x26b430 (0x128e bytes) 000293[1] 1736267659.744320 write 0x4c6ef6 thru 0x4c727a (0x385 bytes) 000294[0] 1736267659.750598 write 0x377cd1 thru 0x379bb1 (0x1ee1 bytes) 000297[1] 1736267659.759997 trunc from 0x64fd43 to 0x56e0d1 (0xe1c71 bytes) 000298[1] 1736267659.770048 mapread 0x3c1ab3 thru 0x3c1bb6 (0x104 bytes) 000299[0] 1736267659.775105 mapread 0x446de1 thru 0x4475b4 (0x7d4 bytes) 000300[0] 1736267659.783886 mapwrite 0x768e81 thru 0x76a96c (0x1aec bytes) 000301[1] 1736267659.813855 mapwrite 0x4126f3 thru 0x412733 (0x41 bytes) 000302[0] 1736267659.833972 mapwrite 0x922f66 thru 0x924d0c (0x1da7 bytes) 000305[0] 1736267659.855995 read 0x41843e thru 0x41a155 (0x1d18 bytes) 000308[0] 1736267659.870132 mapwrite 0x152d51 thru 0x1531d7 (0x487 bytes) 000309[0] 1736267659.884231 trunc from 0x924d0d to 0x48d140 (0x497bcc bytes) 000310[1] 1736267659.891986 mapread 0x39a193 thru 0x39bfa7 (0x1e15 bytes) 000312[1] 1736267659.901681 write 0x3018b4 thru 0x303639 (0x1d86 bytes) 000313[0] 1736267659.908156 mapwrite 0x4f8378 thru 0x4f9685 (0x130e bytes) 000315[0] 1736267659.931169 mapread 0x4f1d50 thru 0x4f34ec (0x179d bytes) 000318[0] 1736267659.940547 mapwrite 0x878841 thru 0x8795ff (0xdbf bytes) 000319[1] 1736267659.958830 write 0x1819b5 thru 0x182e2b (0x1477 bytes) 000321[1] 1736267659.974676 read 0x4ec9eb thru 0x4ed187 (0x79d bytes) 000323[0] 1736267659.994652 mapwrite 0x855e2a thru 0x85634d (0x524 bytes) 000325[1] 1736267660.018132 read 0x501d79 thru 0x503727 (0x19af bytes) 000326[0] 1736267660.020940 read 0x7db32b thru 0x7dd18c (0x1e62 bytes) 000327[0] 1736267660.031994 mapwrite 0x8cd1fa thru 0x8ce569 (0x1370 bytes) 000330[1] 1736267660.055769 mapread 0x88fc99 thru 0x89009f (0x407 bytes) 000331[0] 1736267660.064470 write 0x3a0e8a thru 0x3a1f69 (0x10e0 bytes) 000333[0] 1736267660.069734 mapwrite 0x8a5f82 thru 0x8a710b (0x118a bytes) 000339[0] 1736267660.103271 mapwrite 0x29b2df thru 0x29b36b (0x8d bytes) 000340[1] 1736267660.118394 mapread 0x2887ce thru 0x28a21d (0x1a50 bytes) 000341[0] 1736267660.128421 trunc from 0x8ce56a to 0x3294c8 (0x5a50a1 bytes) 000343[0] 1736267660.134051 mapread 0x320b58 thru 0x3219ed (0xe96 bytes) 000345[0] 1736267660.136210 write 0x78c13e thru 0x78d9c2 (0x1885 bytes) 000347[1] 1736267660.137585 trunc from 0x78d9c3 to 0x33a385 (0x45363d bytes) 000348[1] 1736267660.148026 write 0x91e406 thru 0x920099 (0x1c94 bytes) 000349[1] 1736267660.152284 mapread 0x1fce78 thru 0x1fdd01 (0xe8a bytes) 000351[0] 1736267660.157658 mapread 0x1d2a5d thru 0x1d44c6 (0x1a6a bytes) 000353[1] 1736267660.165147 read 0x5731b5 thru 0x573e75 (0xcc1 bytes) 000357[0] 1736267660.174462 trunc from 0x92009a to 0x804119 (0x11bf80 bytes) 000359[1] 1736267660.186071 mapread 0x5fd17d thru 0x5fe78d (0x1611 bytes) 000364[0] 1736267660.201830 mapread 0x27a520 thru 0x27bbb0 (0x1691 bytes) 000369[1] 1736267660.212265 read 0x5c3f36 thru 0x5c5755 (0x1820 bytes) 000371[1] 1736267660.215570 trunc from 0x80411a to 0x1d9a1a (0x62a6ff bytes) 000375[0] 1736267660.228527 read 0x01d2cb thru 0x01d7d7 (0x50d bytes) 000376[0] 1736267660.237870 read 0x13bc40 thru 0x13c4cb (0x88c bytes) 000378[0] 1736267660.246930 trunc from 0x1d9a1b to 0x2b0eec (0xd74d2 bytes) 000379[1] 1736267660.255050 mapread 0x1ba39c thru 0x1ba49f (0x104 bytes) 000380[1] 1736267660.260050 trunc from 0x2b0eed to 0x0906d3 (0x220819 bytes) 000381[1] 1736267660.270441 read 0x077bf6 thru 0x0782d4 (0x6df bytes) 000383[1] 1736267660.277237 mapread 0x07c38e thru 0x07d466 (0x10d9 bytes) 000389[1] 1736267660.284106 write 0x08309a thru 0x084620 (0x1587 bytes) 000393[1] 1736267660.284933 write 0x66c939 thru 0x66e185 (0x184d bytes) 000394[1] 1736267660.286272 mapread 0x27920c thru 0x27ae6b (0x1c60 bytes) 000395[0] 1736267660.299330 trunc from 0x66e186 to 0x6634e5 (0xaca0 bytes) 000396[0] 1736267660.317252 trunc from 0x6634e6 to 0x5768fb (0xecbea bytes) 000397[0] 1736267660.321100 trunc from 0x5768fc to 0x778f16 (0x20261b bytes) 000398[1] 1736267660.324207 read 0x0d9b89 thru 0x0db0d0 (0x1548 bytes) 000399[1] 1736267660.329148 mapwrite 0x7eebd6 thru 0x7eed9c (0x1c7 bytes) 000400[0] 1736267660.371805 write 0x226cfc thru 0x226f22 (0x227 bytes) 000401[1] 1736267660.379695 read 0x128fca thru 0x12a8fb (0x1932 bytes) 000403[0] 1736267660.386173 read 0x177fd4 thru 0x1795c8 (0x15f5 bytes) 000404[0] 1736267660.414056 write 0x27d6e0 thru 0x27e7ec (0x110d bytes) 000405[1] 1736267660.417219 mapwrite 0x6168b1 thru 0x618663 (0x1db3 bytes) 000407[1] 1736267660.441140 mapread 0x29d6b0 thru 0x29daaa (0x3fb bytes) 000409[1] 1736267660.442508 mapread 0x02a6d9 thru 0x02c3ca (0x1cf2 bytes) 000410[1] 1736267660.445128 read 0x7704f0 thru 0x771928 (0x1439 bytes) 000411[0] 1736267660.449848 write 0x179f6f thru 0x17b707 (0x1799 bytes) 000415[1] 1736267660.467038 read 0x7a6a1b thru 0x7a6e77 (0x45d bytes) 000416[1] 1736267660.476558 trunc from 0x7eed9d to 0x1104d2 (0x6de8ca bytes) 000417[1] 1736267660.484216 read 0x10b90f thru 0x10bb44 (0x236 bytes) 000418[0] 1736267660.488810 mapwrite 0x4cb46a thru 0x4cc0a8 (0xc3f bytes) 000419[1] 1736267660.512987 read 0x2385ad thru 0x239e43 (0x1897 bytes) 000424[1] 1736267660.531552 mapwrite 0x555449 thru 0x556f73 (0x1b2b bytes) 000427[0] 1736267660.569843 write 0x3016c6 thru 0x302ae3 (0x141e bytes) 000432[1] 1736267660.590689 trunc from 0x556f74 to 0x3a7c54 (0x1af31f bytes) 000433[1] 1736267660.600971 trunc from 0x3a7c55 to 0x181664 (0x2265f0 bytes) 000436[0] 1736267660.608762 mapwrite 0x496930 thru 0x497b1e (0x11ef bytes) 000437[0] 1736267660.631683 write 0x2d9057 thru 0x2d9393 (0x33d bytes) 000438[1] 1736267660.635920 mapwrite 0x65588a thru 0x657487 (0x1bfe bytes) 000439[1] 1736267660.660518 mapread 0x0212d1 thru 0x02278c (0x14bc bytes) 000441[0] 1736267660.666742 mapwrite 0x823dee thru 0x824642 (0x855 bytes) 000442[0] 1736267660.691879 mapwrite 0x477e2a thru 0x47871c (0x8f3 bytes) 000443[1] 1736267660.707158 read 0x7ab723 thru 0x7ad158 (0x1a36 bytes) 000446[0] 1736267660.722060 mapwrite 0x9d0b11 thru 0x9d279a (0x1c8a bytes) 000449[1] 1736267660.745033 write 0x1e12f5 thru 0x1e32f4 (0x2000 bytes) 000452[0] 1736267660.757372 read 0x90b966 thru 0x90bf05 (0x5a0 bytes) 000453[0] 1736267660.762756 read 0x20971a thru 0x20a2d5 (0xbbc bytes) 000455[1] 1736267660.790168 mapread 0x83a45f thru 0x83b693 (0x1235 bytes) 000459[0] 1736267660.805903 mapwrite 0x19bb75 thru 0x19be27 (0x2b3 bytes) 000460[1] 1736267660.824939 write 0x001d44 thru 0x0023cd (0x68a bytes) 000461[1] 1736267660.831979 mapwrite 0x347ff4 thru 0x348c4f (0xc5c bytes) 000462[0] 1736267660.847469 write 0x0dc97d thru 0x0dde01 (0x1485 bytes) 000463[0] 1736267660.855573 trunc from 0x9d279b to 0x63bcbb (0x396adf bytes) 000464[0] 1736267660.864488 mapread 0x16fa47 thru 0x170f3f (0x14f9 bytes) 000467[0] 1736267660.873454 mapread 0x04037e thru 0x0408d6 (0x559 bytes) 000468[1] 1736267660.874905 mapwrite 0x5ec7ab thru 0x5ececc (0x722 bytes) 000469[1] 1736267660.891735 write 0x507bb0 thru 0x509537 (0x1988 bytes) 000471[0] 1736267660.898324 write 0x86dec1 thru 0x86faad (0x1bed bytes) 000472[0] 1736267660.901542 mapwrite 0x96de32 thru 0x96e964 (0xb33 bytes) 000473[0] 1736267660.921596 trunc from 0x96e965 to 0x8eb2c3 (0x836a1 bytes) 000474[1] 1736267660.927432 write 0x9805eb thru 0x980fbd (0x9d3 bytes) 000475[0] 1736267660.931993 write 0x9f9275 thru 0x9f9b65 (0x8f1 bytes) 000476[0] 1736267660.935691 trunc from 0x9f9b66 to 0x10ac7e (0x8eeee7 bytes) 000480[1] 1736267660.948201 trunc from 0x10ac7f to 0x7e618e (0x6db510 bytes) 000482[1] 1736267660.959939 read 0x430765 thru 0x431c83 (0x151f bytes) 000483[1] 1736267660.979572 read 0x62af6f thru 0x62c850 (0x18e2 bytes) 000484[1] 1736267660.979882 write 0x2d5d09 thru 0x2d739b (0x1693 bytes) 000485[1] 1736267660.984925 mapread 0x6c5609 thru 0x6c69a1 (0x1399 bytes) 000486[1] 1736267660.988112 trunc from 0x7e618f to 0x5404a7 (0x2a5ce7 bytes) 000488[0] 1736267660.997427 trunc from 0x5404a8 to 0x68e8a5 (0x14e3fe bytes) 000489[0] 1736267661.006250 write 0x6e6ea0 thru 0x6e75d0 (0x731 bytes) 000491[1] 1736267661.013020 mapread 0x6de339 thru 0x6dec8c (0x954 bytes) 000492[1] 1736267661.020020 write 0x97c20c thru 0x97d9cb (0x17c0 bytes) 000494[0] 1736267661.022574 trunc from 0x97d9cc to 0x609f61 (0x373a6a bytes) 000495[0] 1736267661.029689 mapread 0x48cd75 thru 0x48d142 (0x3ce bytes) 000497[0] 1736267661.034992 mapread 0x5298af thru 0x52ae00 (0x1552 bytes) 000499[0] 1736267661.037036 mapread 0x255eae thru 0x25758d (0x16e0 bytes) 000500[1] 1736267661.042035 read 0x15e972 thru 0x15fc26 (0x12b5 bytes) All operations completed A-OK! PASS 16b (5s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 11:34:22 (1736267662) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 11:34:22 (1736267662) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0588237 s, 35.7 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0481002 s, 43.6 MB/s ldlm.namespaces.MGC192.168.201.126@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aa843000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aaac9800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800aaac9800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800aa843000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800aaac9800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.25724 s, 46.5 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.6058 s, 173 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.587839 s, 178 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.13057 s, 92.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.39133 s, 75.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.88318 s, 119 MB/s PASS 16d (10s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 11:34:32 (1736267672) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.054927 s, 191 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282039 s, 37.2 MB/s PASS 16e (1s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 11:34:33 (1736267673) TIMEOUT PASS 16f (22s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 11:34:55 (1736267695) TIMEOUT PASS 16g (21s) == sanityn test 16i: read after truncate file ============ 11:35:16 (1736267716) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (2s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 11:35:18 (1736267718) fail_loc=0x8000030a PASS 17 (3s) == sanityn test 18: mmap sanity check =========================================================================================== 11:35:21 (1736267721) mmap test1: basic mmap operation (PASS, 0.012383s) mmap test2: MAP_PRIVATE not write back (PASS, 0.010416s) mmap test3: concurrent mmap ops on two nodes (PASS, 2.3037s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.0543s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.020476s) mmap test6: check mmap write/read content on two nodes (PASS, 0.026076s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0103s) PASS 18 (8s) == sanityn test 19: test concurrent uncached read races ========================================================================= 11:35:29 (1736267729) oleg126-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg126-client: oleg126-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 11:35:30 (1736267730) PASS 20 (1s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 11:35:31 (1736267731) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) == sanityn test 23: others should see updated atime while another read============================================================== 11:35:33 (1736267733) now is 1736267733 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6846 new atime is 1736267794 PASS 23 (62s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 11:36:35 (1736267795) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 374 114.8K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 375 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 374 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 375 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 8.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 5.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 13.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117974 374 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118071 375 117696 1% /mnt/lustre2[OST:1] filesystem_summary: 235665 369 235296 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 374 114.8K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 375 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 117974 374 117600 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118071 375 117696 1% /mnt/lustre[OST:1] filesystem_summary: 235665 369 235296 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117974 374 117600 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118071 375 117696 1% /mnt/lustre2[OST:1] filesystem_summary: 235665 369 235296 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 8192 3761152 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 5120 3764224 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre2 PASS 24a (2s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 11:36:37 (1736267797) PASS 24b (1s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 11:36:38 (1736267798) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (2s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 11:36:40 (1736267800) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 11:36:41 (1736267801) utime: good mknod times 1736267800 <= 1736267801 <= 1736267801 for /mnt/lustre/f26a utime: good mknod times 1736267800 <= 1736267801 <= 1736267801 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (2s) == sanityn test 26b: sync mtime between ost and mds ====== 11:36:43 (1736267803) PASS 26b (4s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 11:36:47 (1736267807) 0 PASS 30 (4s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 11:36:51 (1736267811) fail_loc=0x314 PASS 31a (2s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 11:36:53 (1736267813) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg126-server mds-ost sync done. sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.201.126@tcp:/lustre 7542784 13312 7523328 1% /mnt/lustre2 PASS 31b (17s) == sanityn test 31r: open-rename(replace) race =========== 11:37:10 (1736267830) -rw-r--r-- 1 root root 0 Jan 7 11:37 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (4s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 11:37:15 (1736267835) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 11:37:15 (1736267835) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (1s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 11:37:16 (1736267836) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 11:37:17 (1736267837) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (0s) == sanityn test 34: no lock timeout under IO ============= 11:37:17 (1736267837) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg126-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa843000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800aaac9800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800aa843000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8800aaac9800.ost_server_uuid in FULL state after 0 sec oleg126-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa843000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800aaac9800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800aa843000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8800aaac9800.ost_server_uuid in IDLE state after 0 sec oleg126-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa843000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800aaac9800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800aa843000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8800aaac9800.ost_server_uuid in FULL state after 0 sec oleg126-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa843000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800aaac9800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800aa843000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8800aaac9800.ost_server_uuid in IDLE state after 0 sec oleg126-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa843000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800aaac9800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800aa843000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8800aaac9800.ost_server_uuid in FULL state after 0 sec oleg126-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa843000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800aaac9800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800aa843000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0001-osc-ffff8800aaac9800.ost_server_uuid in IDLE state after 0 sec PASS 34 (71s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 11:38:28 (1736267908) fail_loc=0x80000317 Race attempt 0 Wait for 15471 15614 for 60 sec... total: 4000 open/close in 7.40 seconds: 540.19 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.201.126@tcp:/lustre 7.2G 15M 7.2G 1% /mnt/lustre 192.168.201.126@tcp:/lustre 7.2G 15M 7.2G 1% /mnt/lustre2 PASS 35 (63s) == sanityn test 36: handle ESTALE/open-unlink correctly == 11:39:31 (1736267971) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg126-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.269059 s, 195 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6846 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.274663 s, 191 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6846 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.277061 s, 189 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6846 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.322714 s, 162 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6846 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.269615 s, 194 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6846 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.267476 s, 196 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6846 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.299195 s, 175 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6846 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.305773 s, 171 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6846 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.276796 s, 189 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6846 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.282529 s, 186 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6846 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7523328) after_dd(7472128) after(7523328) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.301791 s, 174 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6846 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7523328) after_dd(7472128) after(7523328) PASS 36 (227s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 11:43:18 (1736268198) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6846 total: 10000 create in 8.70 seconds: 1149.36 ops/second PASS 37 (11s) == sanityn test 39a: test from 11063 ============================================================================================ 11:43:29 (1736268209) repeat after cancel_lru_locks PASS 39a (2s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 11:43:31 (1736268211) repeat after cancel_lru_locks PASS 39b (2s) == sanityn test 39c: check truncate mtime update ================================================================================ 11:43:33 (1736268213) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 11:43:36 (1736268216) fail_loc=0x411 fail_loc=0 PASS 39d (1s) == sanityn test 40a: pdirops: create vs others ======================================================================== 11:43:37 (1736268217) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (7s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 11:43:44 (1736268224) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (7s) == sanityn test 40c: pdirops: link and others ======================================================================== 11:43:51 (1736268231) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (7s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 11:43:58 (1736268238) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (7s) == sanityn test 40e: pdirops: rename and others ======================================================================== 11:44:05 (1736268245) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (6s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 11:44:11 (1736268251) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (4s) == sanityn test 41b: pdirops: create vs create ======================================================================== 11:44:15 (1736268255) open: File exists Conflict PASS 41b (4s) == sanityn test 41c: pdirops: create vs link ======================================================================== 11:44:19 (1736268259) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (4s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 11:44:23 (1736268263) Conflict PASS 41d (4s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 11:44:27 (1736268267) Conflict PASS 41e (4s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 11:44:32 (1736268272) Conflict PASS 41f (5s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 11:44:36 (1736268276) Conflict PASS 41g (4s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 11:44:40 (1736268280) Conflict PASS 41h (4s) == sanityn test 41i: reint_open: create vs create ======== 11:44:45 (1736268285) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (656s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 11:55:41 (1736268941) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (4s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 11:55:45 (1736268945) open: File exists Conflict PASS 42b (5s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 11:55:50 (1736268950) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (4s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 11:55:54 (1736268954) Conflict PASS 42d (5s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 11:55:59 (1736268959) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (4s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 11:56:03 (1736268963) Conflict PASS 42f (5s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 11:56:08 (1736268968) Conflict PASS 42g (5s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 11:56:13 (1736268973) Conflict PASS 42h (5s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 11:56:18 (1736268978) PASS 43a (15s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 11:56:33 (1736268993) Conflict PASS 43b (5s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 11:56:38 (1736268998) Conflict PASS 43c (4s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 11:56:42 (1736269002) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (5s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 11:56:47 (1736269007) Conflict PASS 43e (5s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 11:56:52 (1736269012) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (4s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 11:56:56 (1736269016) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (4s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 11:57:00 (1736269020) Conflict PASS 43h (4s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 11:57:04 (1736269024) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 11:57:05 (1736269025) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (32s) == sanityn test 43k: unlink vs create ==================== 11:57:37 (1736269057) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (355s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 12:03:32 (1736269412) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Tue Jan 7 12:03:35 EST 2025 PASS 44a (5s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 12:03:37 (1736269417) open: File exists Conflict PASS 44b (4s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 12:03:41 (1736269421) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (4s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 12:03:45 (1736269425) Conflict PASS 44d (5s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 12:03:50 (1736269430) Conflict PASS 44e (4s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 12:03:54 (1736269434) Conflict PASS 44f (4s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 12:03:58 (1736269438) Conflict PASS 44g (5s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 12:04:03 (1736269443) Conflict PASS 44h (4s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 12:04:07 (1736269447) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 12:04:08 (1736269448) PASS 45a (22s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 12:04:30 (1736269470) Conflict PASS 45b (5s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 12:04:35 (1736269475) Conflict PASS 45c (4s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 12:04:39 (1736269479) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (4s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 12:04:43 (1736269483) Conflict PASS 45e (4s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 12:04:48 (1736269488) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (4s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 12:04:52 (1736269492) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (4s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 12:04:56 (1736269496) Conflict PASS 45h (4s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 12:05:01 (1736269501) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (0s) == sanityn test 45j: read vs rename ====================== 12:05:01 (1736269501) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (380s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 12:11:21 (1736269881) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (4s) == sanityn test 46b: pdirops: link vs create ======================================================================== 12:11:25 (1736269885) open: File exists Conflict PASS 46b (5s) == sanityn test 46c: pdirops: link vs link ======================================================================== 12:11:30 (1736269890) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (4s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 12:11:34 (1736269894) Conflict PASS 46d (4s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 12:11:38 (1736269898) Conflict PASS 46e (5s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 12:11:43 (1736269903) Conflict PASS 46f (4s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 12:11:47 (1736269907) Conflict PASS 46g (5s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 12:11:52 (1736269912) Conflict PASS 46h (4s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 12:11:56 (1736269916) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 12:11:57 (1736269917) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 12:11:58 (1736269918) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (1s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 12:11:59 (1736269919) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (0s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 12:11:59 (1736269919) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 12:12:00 (1736269920) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 12:12:01 (1736269921) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (1s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 12:12:02 (1736269922) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 12:12:03 (1736269923) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.00561372 s, 1.8 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (6s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 12:12:10 (1736269930) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00363955 s, 43.7 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (24441) - No such process PASS 51a (4s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 12:12:13 (1736269933) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00277762 s, 369 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129597 s, 80.9 MB/s 3145728 PASS 51b (14s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 12:12:27 (1736269947) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00345834 s, 296 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 12:12:33 (1736269953) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00870822 s, 120 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (2s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 12:12:35 (1736269955) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2435 0x983 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2435 0x983 0 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 12:12:39 (1736269959) llite.lustre-ffff8800aa843000.xattr_cache=1 llite.lustre-ffff8800aaac9800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504963 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 12:12:39.000000000 -0500 Modify: 2025-01-07 12:12:39.000000000 -0500 Change: 2025-01-07 12:12:39.000000000 -0500 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504966 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 12:12:44.000000000 -0500 Modify: 2025-01-07 12:12:45.000000000 -0500 Change: 2025-01-07 12:12:45.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504969 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 12:12:50.000000000 -0500 Modify: 2025-01-07 12:12:50.000000000 -0500 Change: 2025-01-07 12:12:50.000000000 -0500 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 12:12:50.000000000 -0500 Modify: 2025-01-07 12:12:50.000000000 -0500 Change: 2025-01-07 12:12:50.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504972 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 12:12:55.000000000 -0500 Modify: 2025-01-07 12:12:55.000000000 -0500 Change: 2025-01-07 12:12:55.000000000 -0500 Birth: - File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 12:12:55.000000000 -0500 Modify: 2025-01-07 12:12:55.000000000 -0500 Change: 2025-01-07 12:12:55.000000000 -0500 Birth: - PASS 54 (23s) == sanityn test 55a: rename vs unlink target dir ========= 12:13:02 (1736269982) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 12:13:09 (1736269989) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (7s) == sanityn test 55c: rename vs unlink orphan target dir == 12:13:16 (1736269996) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (12s) == sanityn test 55d: rename file vs link ================= 12:13:28 (1736270008) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (9s) == sanityn test 60: Verify data_version behaviour ======== 12:13:37 (1736270017) PASS 60 (1s) == sanityn test 70a: cd directory && rm directory ======== 12:13:38 (1736270018) PASS 70a (2s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 12:13:40 (1736270020) PASS 70b (1s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 12:13:41 (1736270021) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00242203 s, 27.1 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 12:13:42 (1736270022) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00327561 s, 20.0 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 12:13:43 (1736270023) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 12:13:44 (1736270024) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00314707 s, 20.8 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 12:13:45 (1736270025) llite.lustre-ffff8800aa843000.xattr_cache=1 llite.lustre-ffff8800aaac9800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (1s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 12:13:46 (1736270026) llite.lustre-ffff8800aa843000.xattr_cache=1 llite.lustre-ffff8800aaac9800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (2s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 12:13:48 (1736270028) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0023104 s, 443 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0025973 s, 394 kB/s 3085: get lock2 3086: get lock1 3086: done 3086: get lock2 3085: done 3085: get lock1 fcntl cmd 7 failed: Resource deadlock avoided 3085: cannot get lock1: Resource deadlock avoided fcntl cmd 7 failed: Resource deadlock avoided 3086: cannot get lock2: Resource deadlock avoided 3086: exit rc=0 3085: done 3085: exit rc=0 PASS 74 (5s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 12:13:53 (1736270033) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0175362 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104043 s, 101 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 3720 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 12:14:00 (1736270040) opening files: ........ closing files: ........ PASS 76 (12s) == sanityn test 77a: check FIFO NRS policy =============== 12:14:12 (1736270052) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0804677 s, 208 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00637839 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00870396 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0167904 s, 62.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233998 s, 44.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984607 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272397 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199522 s, 52.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131427 s, 79.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0250459 s, 41.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0321514 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196666 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0245919 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0323556 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112429 s, 93.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222707 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142539 s, 73.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0071364 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00744218 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00981838 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00983102 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00920353 s, 114 MB/s , 0.00687239 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00899757 s, 117 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00878795 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00919336 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108417 s, 96.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0100111 s, 105 MB/s , 0.0102183 s, 103 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100449 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00977649 s, 107 MB/s 1048576 bytes (1.0 MB) copied, 0.0109862 s, 95.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113736 s, 92.2 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg126-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) == sanityn test 77b: check CRR-N NRS policy ============== 12:14:14 (1736270054) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0848032 s, 198 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00905036 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00919951 s, 114 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215129 s, 48.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0222532 s, 47.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139352 s, 75.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212537 s, 49.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0262759 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215579 s, 48.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259779 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353623 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315092 s, 33.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.035994 s, 29.1 MB/s , 0.0395672 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0361916 s, 29.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413728 s, 25.3 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=7 ldlm.namespaces.lustre-OST0001-osc-ffff8800aa843000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0492605 s, 21.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00609797 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00480551 s, 218 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00670402 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00560262 s, 187 MB/s 1048576 bytes (1.0 MB) copied, 0.00679058 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107206 s, 97.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111652 s, 93.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00111077 s, 944 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00111682 s, 939 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118259 s, 88.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00910393 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00230579 s, 455 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0012132 s, 864 MB/s , 0.00150185 s, 698 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113567 s, 92.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00121702 s, 862 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg126-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0984199 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00705758 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00834546 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121429 s, 86.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149213 s, 70.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0171092 s, 61.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0149408 s, 70.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00772243 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0214717 s, 48.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248575 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178179 s, 58.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00541713 s, 194 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146996 s, 71.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194836 s, 53.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00730946 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229835 s, 45.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800aa843000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225158 s, 46.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00862 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00815776 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00618017 s, 170 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00741543 s, 141 MB/s , 0.00711157 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00909932 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848057 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00784101 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105586 s, 99.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00799902 s, 131 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00854798 s, 123 MB/s , 0.00927794 s, 113 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00959288 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00959765 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114206 s, 91.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106503 s, 98.5 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg126-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) == sanityn test 77c: check ORR NRS policy ================ 12:14:17 (1736270057) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0944225 s, 178 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00824984 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125709 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177826 s, 59.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025773 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280682 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236526 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297423 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283672 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270402 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280618 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03205 s, 32.7 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800aa843000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317448 s, 33.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0402227 s, 26.1 MB/s , 0.0232075 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00715375 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174951 s, 59.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0084369 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00821527 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010368 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00176553 s, 594 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119031 s, 88.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140711 s, 74.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00921546 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00854557 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00913008 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00851441 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128118 s, 81.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011266 s, 93.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149036 s, 70.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106447 s, 98.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117055 s, 89.6 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118442 s, 88.5 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg126-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0815493 s, 206 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00917409 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121778 s, 86.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206198 s, 50.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174783 s, 60.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289251 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247809 s, 42.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0348842 s, 30.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff8800aa843000.lock_unused_count=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0359233 s, 29.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030685 s, 34.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0380154 s, 27.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0380083 s, 27.6 MB/s , 0.0332691 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0369947 s, 28.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0374634 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464995 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0387779 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00316405 s, 331 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00727917 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00170829 s, 614 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00932345 s, 112 MB/s , 0.00322575 s, 325 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00487003 s, 215 MB/s 1048576 bytes (1.0 MB) copied, 0.00326805 s, 321 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00654576 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00127282 s, 824 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00710523 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00870148 s, 121 MB/s , 0.0087552 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00881249 s, 119 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100593 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0105698 s, 99.2 MB/s 1048576 bytes (1.0 MB) copied, 0.00116942 s, 897 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg126-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0830069 s, 202 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00743898 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136038 s, 77.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202497 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196255 s, 53.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197507 s, 53.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192952 s, 54.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0333292 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0340782 s, 30.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0358001 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0330987 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225288 s, 46.5 MB/s , 0.0225747 s, 46.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037906 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0287619 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295621 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335748 s, 31.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=7 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00827914 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00170354 s, 616 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00716044 s, 146 MB/s , 0.00779582 s, 135 MB/s 1048576 bytes (1.0 MB) copied, 0.00928425 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00783773 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00116142 s, 903 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00166122 s, 631 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00131926 s, 795 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00854114 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00961305 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00976099 s, 107 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00113108 s, 927 MB/s 1048576 bytes (1.0 MB) copied, 0.00739467 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00291295 s, 360 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012975 s, 80.8 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg126-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (4s) == sanityn test 77d: check TRR nrs policy ================ 12:14:21 (1736270061) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0791143 s, 212 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00786768 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012202 s, 85.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016209 s, 64.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00959576 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218157 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210392 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00769041 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00611801 s, 171 MB/s , 0.0203266 s, 51.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153616 s, 68.3 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800aa843000.lock_unused_count=1 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0394007 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0411655 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0373933 s, 28.0 MB/s , 0.0233105 s, 45.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0452576 s, 23.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0415848 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00860066 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00785523 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107809 s, 97.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00602707 s, 174 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103275 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00779911 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00817398 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00125171 s, 838 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00479889 s, 219 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00121847 s, 861 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00900029 s, 117 MB/s 1048576 bytes (1.0 MB) copied, 0.0092047 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00912881 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00978889 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01167 s, 89.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00632324 s, 166 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=3 removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg126-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0923706 s, 182 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112866 s, 92.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123315 s, 85.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219884 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025251 s, 41.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017375 s, 60.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0307115 s, 34.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0288201 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268948 s, 39.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0301548 s, 34.8 MB/s , 0.0354251 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0385606 s, 27.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352552 s, 29.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0347389 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321957 s, 32.6 MB/s 1048576 bytes (1.0 MB) copied, 0.038929 s, 26.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=6 ldlm.namespaces.lustre-OST0001-osc-ffff8800aa843000.lock_unused_count=7 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0379863 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00781473 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00604092 s, 174 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00711006 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00127675 s, 821 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00870793 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00725564 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00448889 s, 234 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00814623 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00941627 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0080918 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00893872 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102918 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00925995 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00498986 s, 210 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00142173 s, 738 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=2 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110026 s, 95.3 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg126-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0885119 s, 190 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00892414 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00966538 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013989 s, 75.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0197828 s, 53.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222585 s, 47.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151365 s, 69.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172923 s, 60.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219741 s, 47.7 MB/s , 0.0334217 s, 31.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360256 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0447656 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0301815 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.039079 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360848 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356189 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.049174 s, 21.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00831741 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00558173 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00580007 s, 181 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00610087 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0076555 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00741062 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0076878 s, 136 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00738687 s, 142 MB/s 1048576 bytes (1.0 MB) copied, 0.00920058 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00122724 s, 854 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00158953 s, 660 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00136737 s, 767 MB/s , 0.00126595 s, 828 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00948337 s, 111 MB/s 1048576 bytes (1.0 MB) copied, 0.00296868 s, 353 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00136384 s, 769 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=1 removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg126-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (3s) == sanityn test 77e: check TBF NID nrs policy ============ 12:14:24 (1736270064) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.201.26@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.127216 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00780742 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121502 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021148 s, 49.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126459 s, 82.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260773 s, 40.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321051 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315148 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280459 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360407 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276115 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0355872 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316917 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0349099 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342002 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0374174 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0323069 s, 32.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00822397 s, 128 MB/s 1048576 bytes (1.0 MB) copied, 0.00556399 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00691009 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00594806 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105131 s, 99.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00891671 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00750463 s, 140 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00843445 s, 124 MB/s , 0.00830335 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00874528 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00889079 s, 118 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00877242 s, 120 MB/s 1048576 bytes (1.0 MB) copied, 0.00914537 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00884938 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00936173 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105207 s, 99.7 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg126-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0730152 s, 230 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00696362 s, 151 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00808575 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119234 s, 87.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118059 s, 88.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0219141 s, 47.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119003 s, 88.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0224456 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272925 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248493 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217462 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.033205 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232233 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254297 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273164 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0223228 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.030874 s, 34.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00603889 s, 174 MB/s , 0.00532912 s, 197 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00887769 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00580132 s, 181 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00691101 s, 152 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0072766 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00755833 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00799017 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00845238 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00826335 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00930157 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00973604 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106875 s, 98.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105846 s, 99.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121539 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120768 s, 86.8 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg126-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.096563 s, 174 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00686328 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0157427 s, 66.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015253 s, 68.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00625754 s, 168 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161232 s, 65.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210123 s, 49.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208547 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.022624 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0252698 s, 41.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148029 s, 70.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025911 s, 40.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183076 s, 57.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0165675 s, 63.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102565 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192692 s, 54.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0262201 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00834479 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00924621 s, 113 MB/s 1048576 bytes (1.0 MB) copied, 0.0102701 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00634854 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103266 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00850495 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105782 s, 99.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00755658 s, 139 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00183859 s, 570 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103162 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00414797 s, 253 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111514 s, 94.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120643 s, 86.9 MB/s 1048576 bytes (1.0 MB) copied, 0.00317264 s, 331 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122067 s, 85.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00294548 s, 356 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg126-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0824912 s, 203 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00774916 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114524 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0099157 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00728541 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213096 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218539 s, 48.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133301 s, 78.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0286792 s, 36.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239198 s, 43.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316876 s, 33.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0330675 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121063 s, 86.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0320262 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220017 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218779 s, 47.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0388723 s, 27.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=2 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00633998 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00789696 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0070428 s, 149 MB/s , 0.0018261 s, 574 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00719563 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00782641 s, 134 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00781997 s, 134 MB/s , 0.0020958 s, 500 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00867812 s, 121 MB/s 1048576 bytes (1.0 MB) copied, 0.0011481 s, 913 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00865398 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00168424 s, 623 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00942732 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00937793 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00128878 s, 814 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105613 s, 99.3 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg126-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (6s) == sanityn test 77f: check TBF JobID nrs policy ========== 12:14:30 (1736270070) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg126-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.119245 s, 141 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00846764 s, 124 MB/s 1048576 bytes (1.0 MB) copied, 0.00870424 s, 120 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256614 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237531 s, 44.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0227192 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279747 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284042 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0351403 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0384552 s, 27.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0419257 s, 25.0 MB/s , 0.0432896 s, 24.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0431445 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0460364 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413188 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0500272 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495221 s, 21.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105085 s, 99.8 MB/s , 0.00928811 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00850201 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00899643 s, 117 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00971388 s, 108 MB/s , 0.00845776 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00991969 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00995862 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out , 0.0106088 s, 98.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0107632 s, 97.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0109408 s, 95.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118133 s, 88.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0118578 s, 88.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0120155 s, 87.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119026 s, 88.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111379 s, 94.1 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg126-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg126-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0815079 s, 206 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104315 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166584 s, 62.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232145 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186615 s, 56.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279959 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273125 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250931 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0302379 s, 34.7 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0312918 s, 33.5 MB/s , 0.026793 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025683 s, 40.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00579095 s, 181 MB/s 1048576 bytes (1.0 MB) copied, 0.0350124 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0254385 s, 41.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126805 s, 82.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100144 s, 105 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00733795 s, 143 MB/s 1048576 bytes (1.0 MB) copied, 0.00902573 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00851386 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00884171 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00855103 s, 123 MB/s 1048576 bytes (1.0 MB) copied, 0.00939901 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121248 s, 86.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00925257 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984551 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116507 s, 90.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109994 s, 95.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100536 s, 104 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0128902 s, 81.3 MB/s , 0.0107754 s, 97.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115841 s, 90.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130031 s, 80.6 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg126-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg126-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.107079 s, 157 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00873956 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00881754 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166985 s, 62.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235876 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207014 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236817 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189423 s, 55.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0260961 s, 40.2 MB/s , 0.0285663 s, 36.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0151213 s, 69.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023403 s, 44.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242933 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174368 s, 60.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378604 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263962 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333636 s, 31.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00852459 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00914557 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00987426 s, 106 MB/s 1048576 bytes (1.0 MB) copied, 0.00882991 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00415051 s, 253 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00158058 s, 663 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00951623 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00928641 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00424221 s, 247 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984176 s, 107 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00895078 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00938337 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105727 s, 99.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105509 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010351 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0013087 s, 801 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg126-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg126-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0903221 s, 186 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00995212 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133677 s, 78.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116423 s, 90.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196808 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103155 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0223719 s, 46.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192714 s, 54.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332202 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0329716 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0194743 s, 53.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315015 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386592 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0407248 s, 25.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0077087 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0442662 s, 23.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800aa843000.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282208 s, 37.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0108138 s, 97.0 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00737609 s, 142 MB/s 1048576 bytes (1.0 MB) copied, 0.00925217 s, 113 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00935887 s, 112 MB/s , 0.00926255 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00976567 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124298 s, 84.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106882 s, 98.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103005 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123529 s, 84.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113424 s, 92.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110769 s, 94.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130531 s, 80.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118999 s, 88.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123156 s, 85.1 MB/s , 0.011999 s, 87.4 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg126-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (5s) == sanityn test 77g: Change TBF type directly ============ 12:14:35 (1736270075) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (4s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 12:14:39 (1736270079) oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg126-client: oleg126-server: ssh exited with exit code 2 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg126-client: oleg126-server: ssh exited with exit code 12 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg126-client: oleg126-server: ssh exited with exit code 12 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg126-client: oleg126-server: ssh exited with exit code 12 PASS 77h (3s) == sanityn test 77i: Change rank of TBF rule ============= 12:14:42 (1736270082) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (6s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 12:14:48 (1736270088) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0775793 s, 216 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00862872 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130714 s, 80.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156196 s, 67.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0165299 s, 63.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138539 s, 75.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0140525 s, 74.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111845 s, 93.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023758 s, 44.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146302 s, 71.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0302336 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154907 s, 67.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.165727 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169314 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159132 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.177145 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166885 s, 6.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00844981 s, 124 MB/s , 0.00226481 s, 463 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00979989 s, 107 MB/s 1048576 bytes (1.0 MB) copied, 0.00765544 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848123 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00987211 s, 106 MB/s , 0.00957011 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00943261 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111202 s, 94.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109336 s, 95.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109413 s, 95.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00957812 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112715 s, 93.0 MB/s , 0.0110883 s, 94.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112147 s, 93.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131143 s, 80.0 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg126-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.91854 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4286 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (49s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 12:15:37 (1736270137) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg126-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.646625 s, 25.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.168502 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188113 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.174828 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.181354 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.185107 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.193641 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188056 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.202647 s, 5.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.365701 s, 2.9 MB/s , 0.363627 s, 2.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.376785 s, 2.8 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.376169 s, 2.8 MB/s , 0.360024 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.363832 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.366937 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.386699 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153099 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15009 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150458 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.15027 s, 7.0 MB/s , 0.153362 s, 6.8 MB/s , 0.153367 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.15173 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151997 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151988 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153292 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153802 s, 6.8 MB/s 1048576 bytes (1.0 MB) copied, 0.153662 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153076 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.154859 s, 6.8 MB/s 1048576 bytes (1.0 MB) copied, 0.154593 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156615 s, 6.7 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg126-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8326 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.959 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg126-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.640993 s, 26.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00697802 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140629 s, 74.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141675 s, 74.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0165791 s, 63.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114193 s, 91.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00658083 s, 159 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0137872 s, 76.1 MB/s , 0.00753808 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.369261 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.373459 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.38715 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.373563 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.385107 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.38441 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.39027 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.400296 s, 2.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.144727 s, 7.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.144073 s, 7.3 MB/s 1048576 bytes (1.0 MB) copied, 0.146234 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145527 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145781 s, 7.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149139 s, 7.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.147268 s, 7.1 MB/s , 0.148362 s, 7.1 MB/s , 0.146957 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151187 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147965 s, 7.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.148566 s, 7.1 MB/s , 0.147948 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149199 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.148865 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150117 s, 7.0 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg126-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8288 s, 5.3 MB/s Write runtime is 20 s, speed is 5.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.966 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (129s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 12:17:46 (1736270266) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg126-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.197642 s, 84.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140694 s, 74.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00941707 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234231 s, 44.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247768 s, 42.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246144 s, 42.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200834 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345663 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187837 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0639613 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0621229 s, 16.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0625387 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0669058 s, 15.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0578329 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0504158 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0813274 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0673918 s, 15.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00902074 s, 116 MB/s , 0.00813098 s, 129 MB/s , 0.0103532 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00906124 s, 116 MB/s , 0.00894472 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0086382 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102656 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00979773 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114044 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114723 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110321 s, 95.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118627 s, 88.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0113015 s, 92.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0128704 s, 81.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118481 s, 88.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145525 s, 72.1 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg126-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93644 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72901 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.26@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.26@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.185649 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126865 s, 82.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229887 s, 45.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270194 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125544 s, 83.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025138 s, 41.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280143 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206328 s, 50.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0204891 s, 51.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0737818 s, 14.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0762577 s, 13.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.173525 s, 6.0 MB/s , 0.168041 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.272114 s, 3.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.264356 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.360447 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.377113 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00788475 s, 133 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114718 s, 91.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008983 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110368 s, 95.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120142 s, 87.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105268 s, 99.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0108743 s, 96.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105508 s, 99.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0117115 s, 89.5 MB/s , 0.0120503 s, 87.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0127906 s, 82.0 MB/s , 0.0121497 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123755 s, 84.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012687 s, 82.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124371 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139711 s, 75.1 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg126-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.91523 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72605 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.26@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg126-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.189887 s, 88.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00639912 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138419 s, 75.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136236 s, 77.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169011 s, 62.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00889793 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140889 s, 74.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150412 s, 69.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155139 s, 67.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0712177 s, 14.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0792023 s, 13.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0730802 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0861656 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0905742 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.090542 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0793325 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.097626 s, 10.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00943513 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00735503 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00882711 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00641238 s, 164 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00954526 s, 110 MB/s , 0.00899799 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0087193 s, 120 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00940923 s, 111 MB/s 1048576 bytes (1.0 MB) copied, 0.00895701 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109147 s, 96.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00902326 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127501 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102445 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105315 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968549 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130216 s, 80.5 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg126-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93617 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.87741 s, 21.5 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg126-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.346209 s, 48.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00999958 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145872 s, 71.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106153 s, 98.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210733 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0136477 s, 76.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159805 s, 65.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185363 s, 56.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00882924 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.162016 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.160203 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.170709 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.177123 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.344315 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.3617 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.369334 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.369663 s, 2.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121716 s, 86.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020876 s, 50.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0200892 s, 52.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101517 s, 103 MB/s 1048576 bytes (1.0 MB) copied, 0.0104705 s, 100 MB/s 1048576 bytes (1.0 MB) copied, 0.0127195 s, 82.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0122655 s, 85.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141056 s, 74.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125067 s, 83.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0199542 s, 52.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266398 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0145339 s, 72.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146881 s, 71.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138581 s, 75.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148019 s, 70.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163099 s, 64.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg126-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7726 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73402 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.90321 s, 21.4 MB/s Write runtime is 5 s, speed is 20.000000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72922 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg126-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.668958 s, 25.1 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145633 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.144138 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.136423 s, 7.7 MB/s 1048576 bytes (1.0 MB) copied, 0.142988 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157209 s, 6.7 MB/s , 0.148986 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155556 s, 6.7 MB/s , 0.138443 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.345347 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.341338 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.361368 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.363337 s, 2.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.371103 s, 2.8 MB/s , 0.378544 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.374016 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.381706 s, 2.7 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00874206 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00626281 s, 167 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00673896 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00739642 s, 142 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00731728 s, 143 MB/s , 0.00907768 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00824308 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00855962 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00900483 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00894621 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00918883 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00955096 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00915285 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0090649 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101066 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121968 s, 86.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg126-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8293 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4305 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg126-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.193317 s, 86.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00933122 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025891 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0166909 s, 62.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193301 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272557 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0283362 s, 37.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265941 s, 39.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238408 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0697803 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0743421 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0793927 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0862057 s, 12.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0874088 s, 12.0 MB/s , 0.0949514 s, 11.0 MB/s , 0.093373 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0845165 s, 12.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00947895 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00898387 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00780775 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107852 s, 97.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00875727 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100155 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00997895 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104201 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102086 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00993132 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010568 s, 99.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112015 s, 93.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102025 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102814 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107051 s, 98.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113662 s, 92.3 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg126-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93254 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72886 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg126-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.342184 s, 49.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00775515 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112097 s, 93.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130207 s, 80.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00973766 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188088 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101111 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114217 s, 91.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00699589 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.172346 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.171216 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.179716 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.17901 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.36659 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.356717 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.374505 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.378121 s, 2.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0118366 s, 88.6 MB/s , 0.00774506 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119148 s, 88.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0079841 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138428 s, 75.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00559495 s, 187 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116512 s, 90.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138855 s, 75.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123343 s, 85.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0137142 s, 76.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150586 s, 69.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130602 s, 80.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134501 s, 78.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0141282 s, 74.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126198 s, 83.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0144305 s, 72.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg126-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77617 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73696 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.76584 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73449 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (356s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 12:23:42 (1736270622) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (2s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 12:23:44 (1736270624) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg126-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.65218 s, 287 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg126-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0071 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 12:24:34 (1736270674) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.104762 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00910836 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113499 s, 92.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00752002 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156813 s, 66.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191022 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270167 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288723 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265103 s, 39.6 MB/s , 0.0148716 s, 70.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271048 s, 38.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266795 s, 39.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.039145 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291612 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249781 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317814 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0325336 s, 32.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800aa843000.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800aa843000.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00802739 s, 131 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00781858 s, 134 MB/s , 0.00821905 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010119 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109834 s, 95.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00971467 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00114293 s, 917 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120928 s, 86.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985274 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115163 s, 91.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101663 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00391667 s, 268 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101098 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100826 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119491 s, 87.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132471 s, 79.2 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg126-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92846 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97034 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.177615 s, 94.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176533 s, 59.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0380929 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265343 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0418311 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386992 s, 27.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177466 s, 59.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0312637 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319005 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0401861 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0443064 s, 23.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.122492 s, 8.6 MB/s , 0.12921 s, 8.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.226437 s, 4.6 MB/s , 0.216644 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.33043 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.320655 s, 3.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146488 s, 71.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0192034 s, 54.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146525 s, 71.6 MB/s , 0.0147868 s, 70.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171695 s, 61.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172089 s, 60.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211978 s, 49.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0169675 s, 61.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171157 s, 61.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0206794 s, 50.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189881 s, 55.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217526 s, 48.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207125 s, 50.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0212524 s, 49.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219646 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0252351 s, 41.6 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg126-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg126-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95238 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97109 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (65s) == sanityn test 77o: Changing rank should not panic ====== 12:25:39 (1736270739) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (3s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 12:25:42 (1736270742) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (30s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 12:26:12 (1736270772) mds.MDS.mdt.nrs_policies=tbf oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg126-client: oleg126-server: ssh exited with exit code 36 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg126-client: oleg126-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (13s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 12:26:25 (1736270785) oleg126-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: Resource temporarily unavailable pdsh@oleg126-client: oleg126-server: ssh exited with exit code 11 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (4s) == sanityn test 79: xattr: intent error ================== 12:26:29 (1736270789) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505431 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 12:26:29.000000000 -0500 Modify: 2025-01-07 12:26:29.000000000 -0500 Change: 2025-01-07 12:26:29.000000000 -0500 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (13s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 12:26:42 (1736270802) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 12:26:43 (1736270803) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (1s) == sanityn test 81a: rename and stat under striped directory ========================================================== 12:26:44 (1736270804) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 12:26:45 (1736270805) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 12:26:46 (1736270806) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 12:26:47 (1736270807) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6846 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6846 PASS 82 (2s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 12:26:49 (1736270809) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 12:26:50 (1736270810) fail_loc=0x8000060b PASS 84 (8s) == sanityn test 90: open/create and unlink striped directory ========================================================== 12:26:58 (1736270818) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (0s) == sanityn test 91: chmod and unlink striped directory === 12:26:58 (1736270818) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 12:26:59 (1736270819) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 12:27:00 (1736270820) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2629 0xa45 0 0 2645 0xa55 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2630 0xa46 0 0 2646 0xa56 0 rc1=0 and rc2=0 PASS 93 (8s) == sanityn test 94: signal vs CP callback race =========== 12:27:08 (1736270828) osc.lustre-OST0000-osc-ffff8800aa843000.idle_timeout=0 osc.lustre-OST0000-osc-ffff8800aaac9800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800aa843000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800aaac9800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.032768 s, 250 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 2.99491 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 4437 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0228468 s, 179 kB/s osc.lustre-OST0000-osc-ffff8800aa843000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800aaac9800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aa843000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aaac9800.idle_timeout=debug PASS 94 (10s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 12:27:18 (1736270838) llite.lustre-ffff8800aa843000.fast_read=0 llite.lustre-ffff8800aaac9800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00284404 s, 4.3 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800aa843000.fast_read=1 llite.lustre-ffff8800aaac9800.fast_read=1 PASS 95b (9s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 12:27:27 (1736270847) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (1s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 12:27:28 (1736270848) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00178099 s, 2.3 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (1s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 12:27:29 (1736270849) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0258439 s, 81.1 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (2s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 12:27:31 (1736270851) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0218154 s, 96.1 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 12:27:33 (1736270853) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.0013551 s, 8.9 MB/s PASS 100e (2s) == sanityn test 101a: Discard DoM data on unlink ========= 12:27:35 (1736270855) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000568158 s, 7.2 MB/s PASS 101a (2s) == sanityn test 101b: Discard DoM data on rename ========= 12:27:37 (1736270857) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000841752 s, 4.9 MB/s PASS 101b (2s) == sanityn test 101c: Discard DoM data on close-unlink === 12:27:39 (1736270859) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0006266 s, 6.5 MB/s PASS 101c (2s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 12:27:42 (1736270862) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1736270863 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1736270865 Finishing test test23 at 1736270865 PASS 103 (6s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 12:27:49 (1736270869) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (0s) == sanityn test 105: Glimpse and lock cancel race ======== 12:27:49 (1736270869) fail_loc=0x416 PASS 105 (48s) == sanityn test 106a: Verify the btime via statx() ======= 12:28:37 (1736270917) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (0s) == sanityn test 106b: Glimpse RPCs test for statx ======== 12:28:38 (1736270918) SKIP: sanityn test_106b statx() only test SKIP 106b (0s) == sanityn test 106c: Verify statx attributes mask ======= 12:28:38 (1736270918) SKIP: sanityn test_106c statx() only test SKIP 106c (1s) == sanityn test 107a: Basic grouplock conflict =========== 12:28:39 (1736270919) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0551937 s, 190 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2635 0xa4b 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6846 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6846 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 12:28:43 (1736270923) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0659649 s, 159 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2652 0xa5c 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6846 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) == sanityn test 108a: lseek: parallel updates ============ 12:28:51 (1736270931) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00142767 s, 7.0 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.15281 s, 2.0 MB/s PASS 108a (6s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 12:28:57 (1736270937) 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 5 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions oleg126-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 10 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions oleg126-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 14 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions oleg126-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 18 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions oleg126-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 21 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions oleg126-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 22 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions oleg126-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 28 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions oleg126-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 32 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 34 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions oleg126-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 35 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions oleg126-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 36 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions oleg126-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 43 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 192.168.201.126@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.126@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg126-server' oleg126-server: oleg126-server.virtnet: executing load_modules_local oleg126-server: Loading modules from /home/green/git/lustre-release/lustre oleg126-server: detected 4 online CPUs by sysfs oleg126-server: Force libcfs to create 2 CPU partitions oleg126-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre Starting client: oleg126-client.virtnet: -o user_xattr,flock oleg126-server@tcp:/lustre /mnt/lustre2 PASS 109 (386s) == sanityn test 110: do not grant another lock on resend ========================================================== 12:35:23 (1736271323) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-07 12:35:23.000000000 -0500 Modify: 2025-01-07 12:35:23.000000000 -0500 Change: 2025-01-07 12:35:23.000000000 -0500 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg126-client.virtnet oleg126-client.virtnet PASS 110 (89s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 12:36:52 (1736271412) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 12:36:53 (1736271413) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (0s) == sanityn test 113: check servers of specified fs ======= 12:36:53 (1736271413) lustre-OST0000-osc-ffff8800a8d8f800 active. lustre-OST0001-osc-ffff8800a8d8f800 active. lustre-OST0000-osc-ffff8800a8d8f800 active. lustre-OST0001-osc-ffff8800a8d8f800 active. MGC192.168.201.126@tcp active. PASS 113 (2s) cleanup: ====================================================== == sanityn test complete, duration 3965 sec ============== 12:36:55 (1736271415) Stopping clients: oleg126-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg126-client.virtnet /mnt/lustre2 opts: