-----============= acceptance-small: sanityn ============----- Wed Jan 15 19:48:22 EST 2025 excepting tests: 27 28 102 skipping tests SLOW=no: 33a Starting client oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 Started clients oleg109-client.virtnet: 192.168.201.109@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt) oleg109-client.virtnet: executing check_config_client /mnt/lustre oleg109-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg109-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800ab31c000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b3d07800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800ab31c000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b3d07800.idle_timeout=debug disable quota as required oleg109-server: oleg109-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 8 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117031 s, 89.6 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [true] running as uid/gid/euid/egid 500/500/500/500, groups: [touch] [/mnt/lustre/d0_runas_test/f6833] == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 19:48:31 (1736988511) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (2s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 19:48:33 (1736988513) -rw-r--r-- 1 root root 0 Jan 15 19:48 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (1s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 19:48:34 (1736988514) -rw-r--r-- 1 root root 0 Jan 15 19:48 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (2s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 19:48:36 (1736988516) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (2s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 19:48:38 (1736988518) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (1s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 19:48:39 (1736988519) total 1 -rwxrwxrwx 1 root root 0 Jan 15 19:48 f2a -rwxrwxrwx 1 root root 0 Jan 15 19:48 f2b total 1 -rwxrwxrwx 1 root root 0 Jan 15 19:48 f2a -rwxrwxrwx 1 root root 0 Jan 15 19:48 f2b running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.00227187 s, 225 kB/s PASS 2e (2s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 19:48:41 (1736988521) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (1s) == sanityn test 2g: check blocks update on sync write ==== 19:48:42 (1736988522) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0702344 s, 29.9 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (2s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 19:48:44 (1736988524) PASS 3 (2s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 19:48:46 (1736988526) PASS 4 (2s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 19:48:48 (1736988528) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (2s) == sanityn test 6: remove of open file on other node ============================================================================ 19:48:50 (1736988530) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (2s) == sanityn test 7: remove of open directory on other node ======================================================================= 19:48:52 (1736988532) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (1s) == sanityn test 8: remove of open special file on other node ==================================================================== 19:48:53 (1736988533) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (2s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 19:48:55 (1736988535) PASS 9a (2s) == sanityn test 9b: append to striped sparse file ======== 19:48:57 (1736988537) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128179 s, 81.8 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0128013 s, 0.2 kB/s Data read (expecting 'foo'): foo PASS 9b (1s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 19:48:58 (1736988538) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00183161 s, 0.5 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00165045 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00162408 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00145296 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00157024 s, 0.6 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00127291 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0011798 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00111063 s, 0.9 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00137576 s, 0.7 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00131608 s, 0.8 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00113896 s, 0.9 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00130313 s, 0.8 kB/s PASS 10a (2s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 19:49:00 (1736988540) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.00398593 s, 771 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0117068 s, 350 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000484917 s, 6.3 MB/s PASS 10b (2s) == sanityn test 11: execution of file opened for write should return error ============================================================== 19:49:02 (1736988542) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.6833 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 273: /mnt/lustre2/d11/f: Text file busy PASS 11 (2s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 19:49:04 (1736988544) warning: '-runtime' deprecated, use '-t runtime' instead - link 2975 (time 1736988554.60 total 10.00 last 297.43) - link 5295 (time 1736988564.60 total 20.01 last 231.94) - link 8066 (time 1736988574.61 total 30.01 last 276.95) - link 10000 (time 1736988582.26 total 37.66 last 252.71) - link 12517 (time 1736988592.26 total 47.66 last 251.67) - link 15794 (time 1736988602.26 total 57.67 last 327.66) - link 18270 (time 1736988612.26 total 67.67 last 247.56) - link 20000 (time 1736988620.84 total 76.24 last 201.82) - link 22810 (time 1736988630.84 total 86.24 last 280.91) - link 25874 (time 1736988640.84 total 96.24 last 306.35) total: 27085 link in 100.00 seconds: 270.85 ops/second using seed 2616584663 running for 100 seconds - stat 10000 (time 1736988548 ; total 3 ; last 3) - stat 20000 (time 1736988550 ; total 5 ; last 2) - stat 30000 (time 1736988553 ; total 8 ; last 3) - stat 40000 (time 1736988556 ; total 11 ; last 3) - stat 50000 (time 1736988558 ; total 13 ; last 2) - stat 60000 (time 1736988562 ; total 17 ; last 4) - stat 70000 (time 1736988564 ; total 19 ; last 2) - stat 80000 (time 1736988567 ; total 22 ; last 3) - stat 90000 (time 1736988569 ; total 24 ; last 2) - stat 100000 (time 1736988572 ; total 27 ; last 3) - stat 110000 (time 1736988574 ; total 29 ; last 2) - stat 120000 (time 1736988578 ; total 33 ; last 4) - stat 130000 (time 1736988580 ; total 35 ; last 2) - stat 140000 (time 1736988583 ; total 38 ; last 3) - stat 150000 (time 1736988585 ; total 40 ; last 2) - stat 160000 (time 1736988588 ; total 43 ; last 3) - stat 170000 (time 1736988591 ; total 46 ; last 3) - stat 180000 (time 1736988593 ; total 48 ; last 2) - stat 190000 (time 1736988597 ; total 52 ; last 4) - stat 200000 (time 1736988603 ; total 58 ; last 6) - stat 210000 (time 1736988611 ; total 66 ; last 8) - stat 220000 (time 1736988619 ; total 74 ; last 8) - stat 230000 (time 1736988627 ; total 82 ; last 8) - stat 240000 (time 1736988634 ; total 89 ; last 7) - stat 250000 (time 1736988640 ; total 95 ; last 6) total: 257455 stats in 100 seconds: 2574.550049 stats/second - unlinked 0 (time 1736988595 ; total 0 ; last 0) - unlinked 10000 (time 1736988647 ; total 52 ; last 52) - unlinked 20000 (time 1736988681 ; total 86 ; last 34) unlink(/mnt/lustre2/lockdir/lockfile27085) error: No such file or directory total: 27084 unlinks in 112 seconds: 241.821426 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (20428) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (20431) - No such process PASS 12 (165s) == sanityn test 13: test directory page revocation ======= 19:51:49 (1736988709) f13 f13 PASS 13 (2s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 19:51:51 (1736988711) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.6833 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 308: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4260: 21826 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (2s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 19:51:53 (1736988713) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4260: 22453 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (2s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 19:51:55 (1736988715) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 323: 23080 Terminated $DIR1/$tdir/sleep 60 PASS 14b (2s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 19:51:57 (1736988717) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 335: 23709 Terminated $DIR1/$tdir/sleep 60 PASS 14c (2s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 19:51:59 (1736988719) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 346: 24338 Terminated $DIR1/$tdir/sleep 60 PASS 14d (2s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 19:52:02 (1736988722) PATH=/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: STRIPECOUNT=2 ORIGFREE=7517184 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7517184kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS zfs Waiting for MDT destroys to complete SKIP 15 (12s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 19:52:13 (1736988733) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.124754 s, 16.8 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0962147 s, 21.8 MB/s Chance of close/open is 1 in 50 Seed set to 8734 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported truncating to largest ever: 0x1fd209 000002[0] 1736988735.194960 trunc from 00000000 to 0x1fd208 (0x1fd209 bytes) 000004[0] 1736988735.200825 write 0x5c02be thru 0x5c761a (0x735d bytes) truncating to largest ever: 0x6912cc 000007[1] 1736988735.209151 trunc from 0x5c761b to 0x6912cb (0xc9cb1 bytes) 000008[0] 1736988735.222638 mapread 0x166884 thru 0x16da30 (0x71ad bytes) 000009[1] 1736988735.234594 read 0x325013 thru 0x3297e9 (0x47d7 bytes) 000010[0] 1736988735.237152 read 0x67faa5 thru 0x68359e (0x3afa bytes) truncating to largest ever: 0x905fe5 000011[1] 1736988735.245140 trunc from 0x6912cc to 0x905fe4 (0x274d19 bytes) 000012[0] 1736988735.255216 read 0x67c6e8 thru 0x6858bf (0x91d8 bytes) 000015[0] 1736988735.270667 mapread 0x707ea0 thru 0x70c08a (0x41eb bytes) 000017[1] 1736988735.283157 mapread 0x821230 thru 0x821474 (0x245 bytes) 000018[1] 1736988735.285655 mapread 0x45dc4e thru 0x4635ae (0x5961 bytes) 000019[1] 1736988735.289759 write 0x359a7e thru 0x36612d (0xc6b0 bytes) 000020[0] 1736988735.298388 mapread 0x90115d thru 0x905fe4 (0x4e88 bytes) 000021[0] 1736988735.309610 trunc from 0x905fe5 to 0x50cd4a (0x3f929a bytes) 000022[0] 1736988735.318060 mapwrite 0x830f3f thru 0x83a173 (0x9235 bytes) 000023[1] 1736988735.356458 mapread 0x10ac99 thru 0x11a46e (0xf7d6 bytes) 000024[0] 1736988735.370293 mapread 0x36a15c thru 0x372448 (0x82ed bytes) 000025[0] 1736988735.384850 trunc from 0x83a174 to 0x291ff2 (0x5a8181 bytes) 000028[1] 1736988735.392939 trunc from 0x291ff3 to 0x8a2c15 (0x610c23 bytes) 000029[0] 1736988735.403991 trunc from 0x8a2c16 to 0x7ebffa (0xb6c1b bytes) 000031[0] 1736988735.420448 mapread 0x242beb thru 0x242f17 (0x32d bytes) 000032[1] 1736988735.422972 read 0x2eb0a4 thru 0x2ee85d (0x37ba bytes) 000033[1] 1736988735.430781 read 0x18c5a9 thru 0x18d9f2 (0x144a bytes) 000034[1] 1736988735.468187 trunc from 0x7ebffb to 0x4718a7 (0x37a753 bytes) 000038[0] 1736988735.517084 trunc from 0x4718a8 to 0x072a56 (0x3fee51 bytes) 000039[0] 1736988735.532106 write 0x28a5ce thru 0x2968cd (0xc300 bytes) 000041[1] 1736988735.543193 mapread 0x2287e4 thru 0x22c394 (0x3bb1 bytes) 000042[0] 1736988735.567269 read 0x16682c thru 0x16d603 (0x6dd8 bytes) 000043[0] 1736988735.581934 trunc from 0x2968ce to 0x207174 (0x8f759 bytes) 000046[1] 1736988735.604262 trunc from 0x207175 to 0x357a8f (0x15091b bytes) 000047[0] 1736988735.620983 read 0x30a963 thru 0x30d0fc (0x279a bytes) 000048[0] 1736988735.640124 mapwrite 0x917e4c thru 0x921786 (0x993b bytes) 000049[1] 1736988735.722138 read 0x81eaaa thru 0x8263d7 (0x792e bytes) 000050[0] 1736988735.751873 write 0x989599 thru 0x98b24f (0x1cb7 bytes) 000053[0] 1736988735.762068 mapwrite 0x349a83 thru 0x35940d (0xf98b bytes) 000054[1] 1736988735.832726 read 0x3166a6 thru 0x31c48b (0x5de6 bytes) 000055[0] 1736988735.864738 mapwrite 0x04f9e8 thru 0x0548a3 (0x4ebc bytes) 000056[1] 1736988735.933963 write 0x53b6a1 thru 0x540e2c (0x578c bytes) 000057[1] 1736988735.954585 mapread 0x1c1104 thru 0x1c5e44 (0x4d41 bytes) 000060[0] 1736988735.991347 read 0x666ed0 thru 0x670e91 (0x9fc2 bytes) 000062[0] 1736988736.014387 mapwrite 0x020d10 thru 0x028483 (0x7774 bytes) 000063[0] 1736988736.064921 mapwrite 0x20d11b thru 0x21ab75 (0xda5b bytes) 000064[1] 1736988736.132141 read 0x897127 thru 0x8a4066 (0xcf40 bytes) 000065[1] 1736988736.147996 read 0x023ae1 thru 0x03291e (0xee3e bytes) 000066[0] 1736988736.190651 read 0x46cd2e thru 0x47328c (0x655f bytes) 000067[1] 1736988736.217407 write 0x1a9ac5 thru 0x1b8984 (0xeec0 bytes) 000068[1] 1736988736.225537 mapwrite 0x25778e thru 0x257dfa (0x66d bytes) 000069[1] 1736988736.274735 trunc from 0x98b250 to 0x05a7da (0x930a75 bytes) 000071[0] 1736988736.299282 write 0x9f510b thru 0x9fffff (0xaef5 bytes) 000072[0] 1736988736.313828 write 0x475d40 thru 0x47866b (0x292c bytes) 000074[1] 1736988736.345890 mapread 0x7c28a3 thru 0x7c6a22 (0x4180 bytes) 000076[0] 1736988736.381861 trunc from 0xa00000 to 0x900ba9 (0xff456 bytes) 000077[1] 1736988736.401482 mapwrite 0x0c1568 thru 0x0c3a54 (0x24ed bytes) 000080[0] 1736988736.466085 read 0x1f2ead thru 0x1fed97 (0xbeeb bytes) truncating to largest ever: 0x908407 000083[1] 1736988736.489768 trunc from 0x900baa to 0x908406 (0x785d bytes) 000084[1] 1736988736.508733 read 0x314087 thru 0x31a077 (0x5ff1 bytes) 000085[0] 1736988736.545947 write 0x55ea73 thru 0x562e66 (0x43f4 bytes) 000087[0] 1736988736.567481 trunc from 0x908407 to 0x6add7c (0x25a68a bytes) 000089[1] 1736988736.586197 read 0x481e9b thru 0x48a6cf (0x8835 bytes) 000090[1] 1736988736.599480 read 0x149d8d thru 0x153e58 (0xa0cc bytes) 000092[0] 1736988736.626623 read 0x10b1a0 thru 0x118064 (0xcec5 bytes) 000094[0] 1736988736.637228 mapwrite 0x84d9bf thru 0x8579a3 (0x9fe5 bytes) 000096[1] 1736988736.705939 mapwrite 0x57e8c1 thru 0x57f9e7 (0x1127 bytes) 000101[0] 1736988736.763537 mapwrite 0x5d3c61 thru 0x5dcef0 (0x9290 bytes) 000102[0] 1736988736.804028 write 0x476dc7 thru 0x47ecbc (0x7ef6 bytes) 000107[0] 1736988736.825229 read 0x827b80 thru 0x82e3b8 (0x6839 bytes) 000108[0] 1736988736.825456 mapwrite 0x915dd4 thru 0x920ea3 (0xb0d0 bytes) 000111[1] 1736988736.869821 write 0x9e62c5 thru 0x9f273e (0xc47a bytes) 000112[1] 1736988736.878743 read 0x190ce7 thru 0x1942d8 (0x35f2 bytes) 000113[0] 1736988736.889464 mapwrite 0x74eb5e thru 0x758584 (0x9a27 bytes) 000115[0] 1736988736.938826 mapwrite 0x8fc130 thru 0x900d7f (0x4c50 bytes) 000116[0] 1736988736.960483 mapread 0x68760d thru 0x696542 (0xef36 bytes) 000118[0] 1736988736.971549 write 0x7fbf40 thru 0x8003da (0x449b bytes) 000119[0] 1736988736.978626 mapread 0x19145e thru 0x1a1445 (0xffe8 bytes) 000120[0] 1736988737.011404 mapread 0x608506 thru 0x60c54a (0x4045 bytes) 000123[1] 1736988737.012109 mapread 0x2d4e4b thru 0x2d9746 (0x48fc bytes) 000125[1] 1736988737.039090 trunc from 0x9f273f to 0x614433 (0x3de30b bytes) 000131[0] 1736988737.067728 mapread 0x577266 thru 0x5796b1 (0x244c bytes) 000132[0] 1736988737.079172 read 0x0c37df thru 0x0d274e (0xef70 bytes) 000134[1] 1736988737.099330 mapread 0x399aa4 thru 0x39b04a (0x15a7 bytes) 000138[1] 1736988737.107573 write 0x319ad4 thru 0x322380 (0x88ad bytes) truncating to largest ever: 0x9c0d62 000140[0] 1736988737.124382 trunc from 0x614434 to 0x9c0d61 (0x3ac92e bytes) 000141[0] 1736988737.143978 trunc from 0x9c0d62 to 0x1daadc (0x7e6285 bytes) 000142[1] 1736988737.148728 mapread 0x18cee3 thru 0x1934e1 (0x65ff bytes) 000144[1] 1736988737.171980 mapread 0x193e40 thru 0x194658 (0x819 bytes) 000145[1] 1736988737.177390 read 0x0843a8 thru 0x087e84 (0x3add bytes) 000147[1] 1736988737.186472 trunc from 0x1daadd to 0x250378 (0x7589c bytes) 000148[1] 1736988737.203375 mapwrite 0x28c67a thru 0x29738e (0xad15 bytes) 000149[0] 1736988737.238845 mapread 0x1c4120 thru 0x1cb407 (0x72e8 bytes) 000151[1] 1736988737.260147 write 0x23d023 thru 0x249dfc (0xcdda bytes) 000152[1] 1736988737.265045 write 0x9a1dc5 thru 0x9ae6a0 (0xc8dc bytes) 000153[1] 1736988737.269339 read 0x84687b thru 0x847f48 (0x16ce bytes) 000155[1] 1736988737.274349 write 0x125601 thru 0x12beec (0x68ec bytes) 000156[1] 1736988737.281987 write 0x9dcc58 thru 0x9e4374 (0x771d bytes) 000157[1] 1736988737.282593 write 0x2b1950 thru 0x2b59ad (0x405e bytes) 000158[1] 1736988737.289981 write 0x45116e thru 0x455194 (0x4027 bytes) 000160[1] 1736988737.301829 mapwrite 0x2ffe81 thru 0x30e39e (0xe51e bytes) 000162[0] 1736988737.351581 mapread 0x2fac3c thru 0x309bb9 (0xef7e bytes) 000167[1] 1736988737.394003 mapread 0x910327 thru 0x91127a (0xf54 bytes) 000168[0] 1736988737.396999 mapwrite 0x95e40f thru 0x96a0b7 (0xbca9 bytes) 000170[0] 1736988737.450547 trunc from 0x9e4375 to 0x3761e4 (0x66e190 bytes) 000173[0] 1736988737.475087 mapwrite 0x482124 thru 0x48a132 (0x800f bytes) 000174[0] 1736988737.509404 mapwrite 0x796d55 thru 0x79cf6d (0x6219 bytes) 000175[1] 1736988737.541852 write 0x0cf746 thru 0x0d9ce0 (0xa59b bytes) 000176[0] 1736988737.555652 mapread 0x4e2f76 thru 0x4f22ab (0xf336 bytes) 000180[1] 1736988737.615132 mapwrite 0x17fc4a thru 0x186a2a (0x6de1 bytes) 000182[1] 1736988737.663805 write 0x815f66 thru 0x8252ad (0xf348 bytes) 000184[1] 1736988737.680428 mapwrite 0x599621 thru 0x5a1ef7 (0x88d7 bytes) 000186[1] 1736988737.731278 mapwrite 0x589928 thru 0x58e99f (0x5078 bytes) 000187[1] 1736988737.769261 trunc from 0x8252ae to 0x16f824 (0x6b5a89 bytes) 000188[1] 1736988737.782610 mapread 0x111d17 thru 0x119483 (0x776d bytes) 000189[0] 1736988737.804842 trunc from 0x16f825 to 0x58e875 (0x41f051 bytes) 000191[0] 1736988737.829008 trunc from 0x58e876 to 0x2d6006 (0x2b886f bytes) truncating to largest ever: 0x9d1a04 000194[0] 1736988737.846413 trunc from 0x2d6007 to 0x9d1a03 (0x6fb9fd bytes) 000195[0] 1736988737.857145 mapwrite 0x8a8b9c thru 0x8aa74d (0x1bb2 bytes) 000197[0] 1736988737.898053 mapwrite 0x198438 thru 0x19e057 (0x5c20 bytes) 000198[0] 1736988737.939782 trunc from 0x9d1a04 to 0x6400dd (0x391926 bytes) 000200[0] 1736988737.953528 mapwrite 0x72cbdc thru 0x73bafe (0xef23 bytes) 000202[1] 1736988738.001950 read 0x02a945 thru 0x02ec42 (0x42fe bytes) 000203[1] 1736988738.014419 mapread 0x481936 thru 0x48a776 (0x8e41 bytes) 000204[0] 1736988738.032440 write 0x17bb67 thru 0x18a862 (0xecfc bytes) 000205[0] 1736988738.035696 read 0x0ae724 thru 0x0bc258 (0xdb35 bytes) 000207[0] 1736988738.064147 mapwrite 0x209fe4 thru 0x20e9f6 (0x4a13 bytes) 000209[1] 1736988738.113232 mapread 0x554990 thru 0x55be16 (0x7487 bytes) 000210[1] 1736988738.143758 write 0x9a1e6b thru 0x9a8e3a (0x6fd0 bytes) 000211[1] 1736988738.152693 trunc from 0x9a8e3b to 0x3ccf9a (0x5dbea0 bytes) 000214[0] 1736988738.165231 read 0x058585 thru 0x05b41b (0x2e97 bytes) 000215[0] 1736988738.175399 write 0x8521a1 thru 0x853e8c (0x1cec bytes) 000222[0] 1736988738.202844 trunc from 0x853e8d to 0x1d6230 (0x67dc5c bytes) 000223[1] 1736988738.209034 trunc from 0x1d6231 to 0x55cb3a (0x38690a bytes) 000224[1] 1736988738.216849 mapwrite 0x8043bf thru 0x80738a (0x2fcc bytes) 000225[1] 1736988738.261478 mapwrite 0x800ca2 thru 0x80941c (0x877b bytes) 000226[1] 1736988738.279043 trunc from 0x80941d to 0x299b57 (0x56f8c5 bytes) 000227[0] 1736988738.292395 read 0x0984d7 thru 0x0a0c4e (0x8778 bytes) 000228[0] 1736988738.304952 mapwrite 0x5dc3ea thru 0x5e7532 (0xb149 bytes) 000234[1] 1736988738.364761 read 0x2a4ee2 thru 0x2af4bc (0xa5db bytes) 000235[0] 1736988738.374432 trunc from 0x5e7533 to 0x298533 (0x34efff bytes) 000237[0] 1736988738.393908 trunc from 0x298534 to 0x2aec7c (0x16749 bytes) 000238[0] 1736988738.405580 read 0x161b67 thru 0x1677d1 (0x5c6b bytes) 000241[1] 1736988738.420599 mapwrite 0x70a5b2 thru 0x712162 (0x7bb1 bytes) 000242[1] 1736988738.484379 mapread 0x455bb2 thru 0x46343d (0xd88c bytes) 000243[0] 1736988738.486566 mapwrite 0x9acf2a thru 0x9bbe03 (0xeeda bytes) 000245[0] 1736988738.562096 read 0x4b74dc thru 0x4c349c (0xbfc1 bytes) 000247[0] 1736988738.588664 read 0x92f867 thru 0x93ba8d (0xc227 bytes) 000248[0] 1736988738.599608 mapwrite 0x3a5d12 thru 0x3af2b2 (0x95a1 bytes) 000251[0] 1736988738.659578 mapread 0x47e291 thru 0x483329 (0x5099 bytes) 000252[1] 1736988738.678935 mapwrite 0x979299 thru 0x98797f (0xe6e7 bytes) 000256[1] 1736988738.783184 trunc from 0x9bbe04 to 0x9b3df4 (0x800f bytes) 000257[1] 1736988738.808157 write 0x4b9a93 thru 0x4bf86f (0x5ddd bytes) 000259[0] 1736988738.826303 trunc from 0x9b3df5 to 0x706eb5 (0x2acf3f bytes) 000260[0] 1736988738.851640 mapread 0x2c3994 thru 0x2cfefe (0xc56b bytes) 000262[0] 1736988738.891926 trunc from 0x706eb6 to 0x596fdb (0x16feda bytes) 000267[0] 1736988738.932329 write 0x1fe5a8 thru 0x200eb4 (0x290d bytes) 000268[0] 1736988738.944478 mapread 0x01bcb8 thru 0x01bef3 (0x23c bytes) 000271[0] 1736988738.954999 write 0x06191d thru 0x069deb (0x84cf bytes) 000272[1] 1736988738.960473 trunc from 0x596fdc to 0x87a618 (0x2e363d bytes) 000273[0] 1736988738.983175 mapwrite 0x0a1c96 thru 0x0a1d1a (0x85 bytes) 000274[0] 1736988739.021121 mapwrite 0x16edcf thru 0x17caa5 (0xdcd7 bytes) 000275[1] 1736988739.091443 mapwrite 0x434fee thru 0x435748 (0x75b bytes) 000276[1] 1736988739.136474 mapread 0x0312e3 thru 0x039ca4 (0x89c2 bytes) 000278[0] 1736988739.175552 mapwrite 0x5dd9eb thru 0x5ddd68 (0x37e bytes) 000281[1] 1736988739.224427 mapwrite 0x9bd262 thru 0x9c0b20 (0x38bf bytes) 000283[0] 1736988739.269585 mapwrite 0x08b7f5 thru 0x094642 (0x8e4e bytes) 000289[0] 1736988739.370446 read 0x30d1a3 thru 0x30e225 (0x1083 bytes) 000290[0] 1736988739.423623 write 0x175f90 thru 0x180a78 (0xaae9 bytes) 000291[1] 1736988739.445784 mapwrite 0x75eb86 thru 0x7664bd (0x7938 bytes) 000294[0] 1736988739.516957 mapwrite 0x8ff3af thru 0x9090ab (0x9cfd bytes) 000296[1] 1736988739.584223 mapwrite 0x1434c5 thru 0x14d71a (0xa256 bytes) 000297[0] 1736988739.643103 trunc from 0x9c0b21 to 0x8e6cf9 (0xd9e27 bytes) 000299[0] 1736988739.664973 write 0x4a74ab thru 0x4b2663 (0xb1b9 bytes) 000302[0] 1736988739.666771 trunc from 0x8e6cfa to 0x06170d (0x8855ec bytes) 000303[0] 1736988739.677223 mapwrite 0x4b54f6 thru 0x4b864c (0x3157 bytes) 000306[1] 1736988739.714006 write 0x722510 thru 0x72d8da (0xb3cb bytes) 000307[0] 1736988739.724985 mapwrite 0x585a7e thru 0x58ae61 (0x53e4 bytes) 000310[1] 1736988739.765463 mapwrite 0x2f3476 thru 0x2fecb7 (0xb842 bytes) 000311[1] 1736988739.804571 write 0x77d37e thru 0x789485 (0xc108 bytes) 000312[0] 1736988739.811100 mapread 0x321b72 thru 0x3227fb (0xc8a bytes) 000313[1] 1736988739.821290 mapread 0x4d0122 thru 0x4d0c24 (0xb03 bytes) 000314[0] 1736988739.826845 mapwrite 0x4f9a60 thru 0x4fa724 (0xcc5 bytes) 000315[1] 1736988739.851682 read 0x47b5ef thru 0x4835fd (0x800f bytes) 000318[0] 1736988739.869665 mapwrite 0x097e4d thru 0x0a7a18 (0xfbcc bytes) 000319[0] 1736988739.926743 mapwrite 0x9cd39a thru 0x9d3106 (0x5d6d bytes) 000320[1] 1736988739.959063 read 0x561061 thru 0x56a18f (0x912f bytes) 000321[0] 1736988739.975189 trunc from 0x9d3107 to 0x6167e8 (0x3bc91e bytes) 000323[1] 1736988739.995845 trunc from 0x6167e9 to 0x981dc0 (0x36b5d8 bytes) 000326[1] 1736988740.020694 mapwrite 0x7bd639 thru 0x7c5550 (0x7f18 bytes) 000327[0] 1736988740.061719 mapread 0x968ebc thru 0x9722fe (0x9443 bytes) 000329[0] 1736988740.095711 mapread 0x028168 thru 0x02c872 (0x470b bytes) 000330[1] 1736988740.113955 mapwrite 0x0980d5 thru 0x09bb6e (0x3a9a bytes) 000331[1] 1736988740.142077 read 0x927a23 thru 0x934b21 (0xd0ff bytes) 000334[1] 1736988740.152122 trunc from 0x981dc1 to 0x2e127e (0x6a0b42 bytes) 000341[0] 1736988740.171436 write 0x5e241e thru 0x5f1290 (0xee73 bytes) 000342[0] 1736988740.180659 mapread 0x0b5024 thru 0x0b7e62 (0x2e3f bytes) 000346[1] 1736988740.204825 write 0x564819 thru 0x567161 (0x2949 bytes) 000349[1] 1736988740.228670 trunc from 0x5f1291 to 0x0151f5 (0x5dc09b bytes) 000350[0] 1736988740.242742 read 0x01004c thru 0x0151f5 (0x51aa bytes) 000352[1] 1736988740.251488 trunc from 0x0151f6 to 0x6840d3 (0x66eede bytes) 000353[1] 1736988740.263385 read 0x4acf15 thru 0x4ad8d2 (0x9be bytes) 000354[1] 1736988740.275809 read 0x13a91d thru 0x1418b1 (0x6f95 bytes) 000359[0] 1736988740.321358 trunc from 0x6840d4 to 0x06a48a (0x619c49 bytes) 000361[1] 1736988740.353813 mapwrite 0x2c3a5a thru 0x2d374b (0xfcf2 bytes) 000362[1] 1736988740.425930 trunc from 0x2d374c to 0x6b08cf (0x3dd184 bytes) 000364[0] 1736988740.439835 mapwrite 0x4b9441 thru 0x4c0518 (0x70d8 bytes) 000365[0] 1736988740.490909 mapread 0x2506d8 thru 0x259baf (0x94d8 bytes) 000367[1] 1736988740.520448 read 0x47f513 thru 0x484a23 (0x5511 bytes) 000368[1] 1736988740.544111 read 0x167584 thru 0x16f1a4 (0x7c21 bytes) 000370[1] 1736988740.568921 mapwrite 0x80ce7f thru 0x80f6bd (0x283f bytes) 000371[1] 1736988740.595069 read 0x547389 thru 0x54d177 (0x5def bytes) 000372[0] 1736988740.595365 mapwrite 0x715c80 thru 0x717757 (0x1ad8 bytes) 000373[0] 1736988740.629173 trunc from 0x80f6be to 0x8d9201 (0xc9b44 bytes) 000374[0] 1736988740.643413 mapread 0x3479ef thru 0x34cd20 (0x5332 bytes) 000377[1] 1736988740.683690 trunc from 0x8d9202 to 0x38243a (0x556dc7 bytes) 000378[1] 1736988740.714202 mapread 0x15406a thru 0x15589f (0x1836 bytes) 000379[1] 1736988740.725971 write 0x1d45c4 thru 0x1e0876 (0xc2b3 bytes) 000381[0] 1736988740.732901 read 0x21cd3b thru 0x224308 (0x75ce bytes) 000382[1] 1736988740.754006 write 0x2b935d thru 0x2ba622 (0x12c6 bytes) 000383[0] 1736988740.764278 write 0x0f030f thru 0x0f6c4b (0x693d bytes) 000385[1] 1736988740.788873 trunc from 0x38243b to 0x6d1f94 (0x34fb5a bytes) 000391[1] 1736988740.827801 mapwrite 0x411770 thru 0x41fe57 (0xe6e8 bytes) 000392[0] 1736988740.879394 mapwrite 0x98345d thru 0x98e367 (0xaf0b bytes) 000393[1] 1736988740.963146 mapread 0x616152 thru 0x6249c7 (0xe876 bytes) 000397[0] 1736988740.999681 trunc from 0x98e368 to 0x068c51 (0x925716 bytes) 000400[0] 1736988741.013294 mapread 0x033a08 thru 0x03fd67 (0xc360 bytes) 000402[1] 1736988741.036828 mapread 0x03be5c thru 0x03e21d (0x23c2 bytes) 000404[0] 1736988741.052708 read 0x0509eb thru 0x057560 (0x6b76 bytes) 000406[0] 1736988741.064009 mapread 0x0299ad thru 0x03176c (0x7dc0 bytes) 000407[1] 1736988741.089559 write 0x7dc1ab thru 0x7e600d (0x9e63 bytes) 000409[1] 1736988741.096985 read 0x062832 thru 0x066921 (0x40f0 bytes) 000410[0] 1736988741.118680 trunc from 0x7e600e to 0x43c440 (0x3a9bcd bytes) 000413[1] 1736988741.158001 mapread 0x2aa5f0 thru 0x2af53a (0x4f4b bytes) 000414[1] 1736988741.183888 read 0x35c9db thru 0x3681a0 (0xb7c6 bytes) 000416[1] 1736988741.199415 write 0x3aaf3a thru 0x3ab45b (0x522 bytes) 000417[1] 1736988741.215423 mapread 0x2e9642 thru 0x2f278a (0x9149 bytes) 000418[0] 1736988741.244929 write 0x472fc7 thru 0x475417 (0x2451 bytes) 000419[0] 1736988741.256840 mapread 0x1d1b1d thru 0x1d56da (0x3bbe bytes) 000421[0] 1736988741.303294 read 0x0d9b84 thru 0x0e7ae6 (0xdf63 bytes) 000422[1] 1736988741.303890 trunc from 0x475418 to 0x4ce4e1 (0x590ca bytes) 000423[1] 1736988741.335780 read 0x1a8069 thru 0x1a9465 (0x13fd bytes) 000424[1] 1736988741.369407 trunc from 0x4ce4e2 to 0x5b652e (0xe804d bytes) 000425[0] 1736988741.385955 write 0x72b9c7 thru 0x7361e8 (0xa822 bytes) 000429[0] 1736988741.423008 read 0x1000a3 thru 0x10a550 (0xa4ae bytes) 000431[1] 1736988741.448487 mapwrite 0x234888 thru 0x23bb4c (0x72c5 bytes) 000434[1] 1736988741.498168 read 0x5e7265 thru 0x5e7b96 (0x932 bytes) 000435[1] 1736988741.528184 write 0x988158 thru 0x990757 (0x8600 bytes) 000436[0] 1736988741.540914 read 0x2a9b74 thru 0x2b0966 (0x6df3 bytes) 000437[1] 1736988741.557085 mapread 0x353001 thru 0x353b84 (0xb84 bytes) 000441[0] 1736988741.581913 mapread 0x90ef70 thru 0x912793 (0x3824 bytes) 000442[0] 1736988741.634265 trunc from 0x990758 to 0x9130b0 (0x7d6a7 bytes) 000443[1] 1736988741.660564 trunc from 0x9130b1 to 0x90f2bb (0x3df5 bytes) 000445[0] 1736988741.684411 trunc from 0x90f2bc to 0x135e11 (0x7d94aa bytes) 000447[0] 1736988741.700610 mapwrite 0x228dca thru 0x236123 (0xd35a bytes) 000450[1] 1736988741.763942 mapread 0x03cd84 thru 0x04343a (0x66b7 bytes) 000451[0] 1736988741.797766 write 0x4335ae thru 0x435c55 (0x26a8 bytes) 000452[1] 1736988741.804076 trunc from 0x435c56 to 0x5a9f84 (0x17432f bytes) 000455[0] 1736988741.837694 trunc from 0x5a9f85 to 0x265660 (0x344924 bytes) 000459[1] 1736988741.868874 mapwrite 0x0d1677 thru 0x0d5816 (0x41a0 bytes) 000462[1] 1736988741.928359 read 0x039a83 thru 0x03b537 (0x1ab5 bytes) 000463[0] 1736988741.944043 mapread 0x177376 thru 0x17a208 (0x2e93 bytes) 000464[0] 1736988741.961973 mapread 0x195f7c thru 0x199ff0 (0x4075 bytes) 000466[0] 1736988741.990006 mapwrite 0x9711d9 thru 0x974c26 (0x3a4e bytes) 000470[1] 1736988742.042625 trunc from 0x974c27 to 0x7a493e (0x1d02e8 bytes) 000471[0] 1736988742.060750 mapwrite 0x6d7eaf thru 0x6d9b25 (0x1c77 bytes) 000472[1] 1736988742.106403 mapwrite 0x856acc thru 0x85872d (0x1c62 bytes) 000474[1] 1736988742.189501 trunc from 0x85872e to 0x5df4db (0x279252 bytes) 000476[0] 1736988742.205915 read 0x3de5de thru 0x3e0588 (0x1fab bytes) 000481[0] 1736988742.218401 mapwrite 0x0888e5 thru 0x08f9a9 (0x70c5 bytes) 000484[0] 1736988742.271418 write 0x4e0a37 thru 0x4e5cc4 (0x528e bytes) 000485[1] 1736988742.271945 write 0x78b5bc thru 0x799837 (0xe27c bytes) 000487[0] 1736988742.280280 mapread 0x434da1 thru 0x441e79 (0xd0d9 bytes) 000488[1] 1736988742.284145 trunc from 0x799838 to 0x645403 (0x154434 bytes) 000490[0] 1736988742.296430 write 0x3f64ad thru 0x3febc6 (0x871a bytes) 000491[1] 1736988742.302953 read 0x56f53c thru 0x573ea4 (0x4969 bytes) 000492[1] 1736988742.316659 write 0x9f8f15 thru 0x9fffff (0x70eb bytes) 000493[1] 1736988742.321540 mapread 0x2e424f thru 0x2e785a (0x360c bytes) 000495[0] 1736988742.340309 write 0x8601ea thru 0x86e30c (0xe123 bytes) 000496[0] 1736988742.349345 trunc from 0xa00000 to 0x22bd35 (0x7d42ca bytes) 000499[1] 1736988742.362473 mapread 0x1d41a9 thru 0x1d7974 (0x37cc bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 8742 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[0] 1736988742.615881 write 0x1ec000 thru 0x1fa4c5 (0xe4c6 bytes) 000003[1] 1736988742.629940 mapwrite 0x3ff000 thru 0x402e9f (0x3ea0 bytes) 000005[1] 1736988742.673602 mapwrite 0x8c8000 thru 0x8d01dc (0x81dd bytes) 000007[1] 1736988742.701537 mapread 0x5ad000 thru 0x5bac3e (0xdc3f bytes) 000008[0] 1736988742.716109 mapread 0x34d000 thru 0x3501b2 (0x31b3 bytes) 000011[0] 1736988742.730357 write 0x91c000 thru 0x91c3f3 (0x3f4 bytes) 000012[0] 1736988742.734753 write 0x242000 thru 0x251214 (0xf215 bytes) 000013[0] 1736988742.737614 mapread 0x7ce000 thru 0x7ce421 (0x422 bytes) 000017[1] 1736988742.746145 mapread 0x81d000 thru 0x8295ec (0xc5ed bytes) 000018[1] 1736988742.766600 mapwrite 0x9ee000 thru 0x9f28a8 (0x48a9 bytes) 000021[1] 1736988742.817758 mapread 0x931000 thru 0x938741 (0x7742 bytes) 000022[0] 1736988742.828124 read 0x135000 thru 0x135c7f (0xc80 bytes) truncating to largest ever: 0x35825a 000024[1] 1736988742.836818 trunc from 0x9f28a9 to 0x358259 (0x69a64f bytes) 000026[1] 1736988742.848511 mapwrite 0x45f000 thru 0x461729 (0x272a bytes) truncating to largest ever: 0x8719be 000030[0] 1736988742.901764 trunc from 0x46172a to 0x8719bd (0x410294 bytes) 000031[1] 1736988742.911584 mapread 0x28e000 thru 0x298d6c (0xad6d bytes) 000033[0] 1736988742.923678 mapwrite 0x4e8000 thru 0x4ed09b (0x509c bytes) 000038[0] 1736988742.945181 mapwrite 0x8b8000 thru 0x8c399f (0xb9a0 bytes) 000039[0] 1736988742.968342 write 0x8e2000 thru 0x8e72d8 (0x52d9 bytes) 000040[1] 1736988742.969803 read_OD 0x363000 thru 0x366fff (0x4000 bytes) 000043[0] 1736988742.977382 mapwrite 0x3c1000 thru 0x3c394f (0x2950 bytes) 000044[0] 1736988742.991808 read 0x449000 thru 0x4494cf (0x4d0 bytes) 000049[1] 1736988743.013872 mapread 0x46f000 thru 0x474cec (0x5ced bytes) 000050[0] 1736988743.025067 read 0x59a000 thru 0x5a57b0 (0xb7b1 bytes) 000051[0] 1736988743.037367 read 0x046000 thru 0x0472ff (0x1300 bytes) 000052[0] 1736988743.056327 write 0x16f000 thru 0x17ea1e (0xfa1f bytes) 000053[0] 1736988743.069769 trunc from 0x8e72d9 to 0x20e6f2 (0x6d8be6 bytes) 000056[1] 1736988743.080406 mapwrite 0x238000 thru 0x2401de (0x81df bytes) 000057[0] 1736988743.101745 mapread 0x229000 thru 0x231fbf (0x8fc0 bytes) 000065[1] 1736988743.115589 mapread 0x122000 thru 0x12e83d (0xc83e bytes) 000067[1] 1736988743.121336 read_OD 0x185000 thru 0x187fff (0x3000 bytes) 000068[0] 1736988743.122632 mapread 0x1a1000 thru 0x1a968d (0x868e bytes) 000069[1] 1736988743.131207 read_OD 0x201000 thru 0x202fff (0x2000 bytes) 000070[1] 1736988743.132393 mapwrite 0x0de000 thru 0x0e01b3 (0x21b4 bytes) 000071[0] 1736988743.151358 mapwrite 0x96b000 thru 0x9790dc (0xe0dd bytes) 000072[1] 1736988743.175153 read_OD 0x7e4000 thru 0x7e8fff (0x5000 bytes) 000074[1] 1736988743.183057 read_OD 0x470000 thru 0x477fff (0x8000 bytes) 000077[0] 1736988743.185095 read 0x264000 thru 0x26d1d5 (0x91d6 bytes) 000079[1] 1736988743.203834 mapwrite 0x368000 thru 0x372de5 (0xade6 bytes) 000084[1] 1736988743.267852 read_OD 0x883000 thru 0x88ffff (0xd000 bytes) truncating to largest ever: 0x8f512a 000085[0] 1736988743.279601 trunc from 0x9790dd to 0x8f5129 (0x83fb3 bytes) 000086[0] 1736988743.289079 mapwrite 0x389000 thru 0x3923e0 (0x93e1 bytes) 000088[1] 1736988743.325426 read_OD 0x149000 thru 0x14afff (0x2000 bytes) 000089[1] 1736988743.330353 trunc from 0x8f512a to 0x121854 (0x7d38d5 bytes) 000090[1] 1736988743.335880 read_OD 0x018000 thru 0x01ffff (0x8000 bytes) 000091[0] 1736988743.340533 write 0x5dc000 thru 0x5e9fa9 (0xdfaa bytes) 000092[0] 1736988743.349182 read 0x473000 thru 0x476b53 (0x3b54 bytes) 000093[0] 1736988743.356996 mapwrite 0x279000 thru 0x281b78 (0x8b79 bytes) 000094[0] 1736988743.382996 mapwrite 0x412000 thru 0x42112a (0xf12b bytes) 000097[0] 1736988743.422298 read 0x379000 thru 0x37f692 (0x6693 bytes) 000099[1] 1736988743.436119 read_OD 0x2cf000 thru 0x2dcfff (0xe000 bytes) 000100[1] 1736988743.443345 trunc from 0x5e9faa to 0x449135 (0x1a0e74 bytes) 000101[1] 1736988743.450302 write_OD 0x5d9000 thru 0x5e2fff (0xa000 bytes) 000103[0] 1736988743.469059 mapwrite 0x4d2000 thru 0x4d9f24 (0x7f25 bytes) 000104[1] 1736988743.506792 trunc from 0x5e3000 to 0x36f381 (0x273c7e bytes) 000105[0] 1736988743.518573 read 0x00c000 thru 0x00d03f (0x1040 bytes) 000106[0] 1736988743.539680 trunc from 0x36f382 to 0x76525b (0x3f5eda bytes) 000107[0] 1736988743.553904 trunc from 0x76525c to 0x36a054 (0x3fb207 bytes) 000108[1] 1736988743.557894 read_OD 0x19a000 thru 0x1a2fff (0x9000 bytes) 000113[0] 1736988743.571413 write 0x2bd000 thru 0x2bf612 (0x2613 bytes) 000115[0] 1736988743.584679 read 0x322000 thru 0x327ff3 (0x5ff4 bytes) 000118[1] 1736988743.597889 mapwrite 0x2f7000 thru 0x2f769b (0x69c bytes) 000119[1] 1736988743.646004 trunc from 0x36a055 to 0x1e6060 (0x183ff4 bytes) 000120[0] 1736988743.655373 read 0x0a8000 thru 0x0ae410 (0x6411 bytes) 000126[1] 1736988743.675222 trunc from 0x1e6061 to 0x2fcc61 (0x116c01 bytes) 000128[0] 1736988743.695836 write 0x6de000 thru 0x6e1a44 (0x3a45 bytes) 000132[1] 1736988743.708599 mapwrite 0x362000 thru 0x368a1e (0x6a1f bytes) truncating to largest ever: 0x9c1e52 000133[1] 1736988743.787393 trunc from 0x6e1a45 to 0x9c1e51 (0x2e040d bytes) 000135[1] 1736988743.801592 mapwrite 0x664000 thru 0x665363 (0x1364 bytes) 000136[1] 1736988743.825155 write_OD 0x5b9000 thru 0x5c6fff (0xe000 bytes) 000137[0] 1736988743.847476 mapread 0x633000 thru 0x63e53a (0xb53b bytes) 000138[1] 1736988743.891590 mapwrite 0x134000 thru 0x134d06 (0xd07 bytes) 000139[1] 1736988743.920935 mapwrite 0x588000 thru 0x58b4ba (0x34bb bytes) 000140[1] 1736988743.936726 mapread 0x289000 thru 0x29660e (0xd60f bytes) 000141[0] 1736988743.958708 mapread 0x962000 thru 0x968be2 (0x6be3 bytes) 000142[0] 1736988743.972042 mapread 0x50e000 thru 0x5187f6 (0xa7f7 bytes) 000145[0] 1736988743.989226 mapwrite 0x2b2000 thru 0x2bb630 (0x9631 bytes) 000147[0] 1736988744.025714 mapread 0x113000 thru 0x119445 (0x6446 bytes) 000149[0] 1736988744.041626 trunc from 0x9c1e52 to 0x48dced (0x534164 bytes) 000151[0] 1736988744.055709 mapread 0x3d5000 thru 0x3d9257 (0x4258 bytes) 000154[0] 1736988744.073949 mapwrite 0x2f8000 thru 0x2fd442 (0x5443 bytes) 000156[1] 1736988744.101061 mapwrite 0x25b000 thru 0x25e9d5 (0x39d6 bytes) 000157[0] 1736988744.125760 read 0x037000 thru 0x03bad1 (0x4ad2 bytes) 000159[0] 1736988744.149138 mapwrite 0x390000 thru 0x39221a (0x221b bytes) 000160[0] 1736988744.201744 mapread 0x404000 thru 0x40be6d (0x7e6e bytes) 000161[0] 1736988744.218057 mapread 0x3a3000 thru 0x3b07f0 (0xd7f1 bytes) 000162[0] 1736988744.248810 trunc from 0x48dcee to 0x48d47d (0x870 bytes) 000164[0] 1736988744.267582 trunc from 0x48d47e to 0x84a8fc (0x3bd47f bytes) 000165[1] 1736988744.277113 write_OD 0x3a9000 thru 0x3adfff (0x5000 bytes) 000172[1] 1736988744.304505 mapread 0x30d000 thru 0x310f32 (0x3f33 bytes) 000173[1] 1736988744.313839 read_OD 0x5fe000 thru 0x5fefff (0x1000 bytes) 000174[0] 1736988744.316474 trunc from 0x84a8fd to 0x406794 (0x444168 bytes) 000175[1] 1736988744.335708 trunc from 0x406795 to 0x27cb60 (0x189c34 bytes) 000176[0] 1736988744.346581 read 0x187000 thru 0x18f077 (0x8078 bytes) 000180[0] 1736988744.369006 write 0x99c000 thru 0x9a5ca0 (0x9ca1 bytes) 000181[1] 1736988744.382647 mapwrite 0x21e000 thru 0x21f211 (0x1212 bytes) 000182[1] 1736988744.423748 read_OD 0x6e2000 thru 0x6e9fff (0x8000 bytes) 000183[1] 1736988744.433383 read_OD 0x52a000 thru 0x530fff (0x7000 bytes) 000185[0] 1736988744.436455 read 0x6e2000 thru 0x6e8658 (0x6659 bytes) 000187[1] 1736988744.449876 write_OD 0x702000 thru 0x70ffff (0xe000 bytes) 000189[0] 1736988744.466280 read 0x92a000 thru 0x92c81f (0x2820 bytes) 000190[0] 1736988744.470772 mapwrite 0x6a4000 thru 0x6a4b01 (0xb02 bytes) 000192[0] 1736988744.487633 trunc from 0x9a5ca1 to 0x8547b1 (0x1514ef bytes) 000193[0] 1736988744.492538 mapwrite 0x6e0000 thru 0x6ebe87 (0xbe88 bytes) 000194[1] 1736988744.532540 write_OD 0x7d8000 thru 0x7dafff (0x3000 bytes) 000196[1] 1736988744.555611 write_OD 0x360000 thru 0x366fff (0x7000 bytes) 000198[0] 1736988744.580007 trunc from 0x8547b2 to 0x67cca5 (0x1d7b0c bytes) 000199[0] 1736988744.591366 write 0x6e2000 thru 0x6ea788 (0x8789 bytes) 000201[0] 1736988744.593461 read 0x0e2000 thru 0x0e4084 (0x2085 bytes) 000202[1] 1736988744.636289 read_OD 0x2ec000 thru 0x2f1fff (0x6000 bytes) 000204[0] 1736988744.662649 mapwrite 0x7aa000 thru 0x7af365 (0x5366 bytes) 000205[0] 1736988744.705622 trunc from 0x7af366 to 0x30e5d2 (0x4a0d93 bytes) 000213[1] 1736988744.745078 read_OD 0x056000 thru 0x05cfff (0x7000 bytes) 000214[1] 1736988744.753756 write_OD 0x698000 thru 0x6a1fff (0xa000 bytes) 000215[1] 1736988744.771610 trunc from 0x6a2000 to 0x3196af (0x388950 bytes) 000219[0] 1736988744.786833 mapwrite 0x5d6000 thru 0x5d9fef (0x3ff0 bytes) 000220[0] 1736988744.819015 mapwrite 0x532000 thru 0x535681 (0x3682 bytes) 000222[1] 1736988744.845143 write_OD 0x07f000 thru 0x088fff (0xa000 bytes) 000224[0] 1736988744.870478 read 0x45c000 thru 0x464ed2 (0x8ed3 bytes) 000226[0] 1736988744.886664 trunc from 0x5d9ff0 to 0x559fc4 (0x8002b bytes) 000227[1] 1736988744.907585 write_OD 0x234000 thru 0x23bfff (0x8000 bytes) 000229[0] 1736988744.929648 read 0x39a000 thru 0x3a978f (0xf790 bytes) 000230[1] 1736988744.944676 mapread 0x246000 thru 0x25525b (0xf25c bytes) 000236[0] 1736988744.974401 mapread 0x0e4000 thru 0x0e4c04 (0xc05 bytes) 000237[0] 1736988744.976671 mapread 0x458000 thru 0x45c8ff (0x4900 bytes) 000238[0] 1736988744.976989 trunc from 0x559fc5 to 0x6737e5 (0x119821 bytes) 000241[1] 1736988745.006843 write_OD 0x414000 thru 0x420fff (0xd000 bytes) 000243[0] 1736988745.026974 mapread 0x168000 thru 0x16e209 (0x620a bytes) 000247[0] 1736988745.042357 write 0x953000 thru 0x95ac63 (0x7c64 bytes) 000251[1] 1736988745.061811 write_OD 0x8fb000 thru 0x8fbfff (0x1000 bytes) 000252[1] 1736988745.084582 write_OD 0x47b000 thru 0x484fff (0xa000 bytes) 000253[1] 1736988745.109896 mapread 0x4b9000 thru 0x4c1631 (0x8632 bytes) 000254[0] 1736988745.130036 trunc from 0x95ac64 to 0x459faf (0x500cb4 bytes) 000258[0] 1736988745.158839 mapread 0x1ed000 thru 0x1fba68 (0xea69 bytes) 000259[1] 1736988745.191457 mapwrite 0x7ae000 thru 0x7b584a (0x784b bytes) 000260[1] 1736988745.254347 write_OD 0x177000 thru 0x17bfff (0x5000 bytes) 000261[0] 1736988745.279831 mapwrite 0x16a000 thru 0x1782cd (0xe2ce bytes) 000262[1] 1736988745.346691 trunc from 0x7b584b to 0x287479 (0x52e3d1 bytes) 000267[1] 1736988745.374178 mapwrite 0x820000 thru 0x82b5c4 (0xb5c5 bytes) 000269[0] 1736988745.443870 trunc from 0x82b5c5 to 0x5cd03b (0x25e589 bytes) 000270[0] 1736988745.456449 read 0x2ee000 thru 0x2f4581 (0x6582 bytes) 000272[1] 1736988745.487054 trunc from 0x5cd03c to 0x160a57 (0x46c5e4 bytes) 000277[1] 1736988745.527214 trunc from 0x160a58 to 0x0f8916 (0x68141 bytes) 000279[0] 1736988745.535921 read 0x043000 thru 0x045581 (0x2582 bytes) 000282[1] 1736988745.553847 trunc from 0x0f8917 to 0x47fa8a (0x387174 bytes) 000285[0] 1736988745.576588 trunc from 0x47fa8b to 0x620574 (0x1a0aea bytes) 000287[0] 1736988745.597861 mapwrite 0x1ec000 thru 0x1fbc13 (0xfc14 bytes) 000288[0] 1736988745.648985 write 0x5e9000 thru 0x5f67b5 (0xd7b6 bytes) 000291[1] 1736988745.658611 mapread 0x509000 thru 0x51341f (0xa420 bytes) 000293[0] 1736988745.682697 read 0x2c1000 thru 0x2cf052 (0xe053 bytes) 000295[0] 1736988745.697407 write 0x058000 thru 0x05f850 (0x7851 bytes) 000300[0] 1736988745.711774 write 0x203000 thru 0x2086e6 (0x56e7 bytes) 000301[0] 1736988745.716222 trunc from 0x620575 to 0x4675ad (0x1b8fc7 bytes) 000302[0] 1736988745.720010 read 0x0c0000 thru 0x0c0f36 (0xf37 bytes) 000303[0] 1736988745.738446 read 0x43a000 thru 0x441756 (0x7757 bytes) 000304[1] 1736988745.738705 mapwrite 0x428000 thru 0x42e5f3 (0x65f4 bytes) 000305[1] 1736988745.768873 mapwrite 0x00d000 thru 0x012be0 (0x5be1 bytes) 000308[0] 1736988745.789878 write 0x601000 thru 0x603e3d (0x2e3e bytes) 000312[1] 1736988745.807341 trunc from 0x603e3e to 0x52b3d0 (0xd8a6d bytes) 000315[1] 1736988745.818279 trunc from 0x52b3d1 to 0x83ee13 (0x313a43 bytes) 000320[0] 1736988745.833617 read 0x35a000 thru 0x35a7ff (0x800 bytes) 000321[0] 1736988745.852935 write 0x718000 thru 0x719704 (0x1705 bytes) 000322[0] 1736988745.865185 write 0x770000 thru 0x77e69f (0xe6a0 bytes) 000325[1] 1736988745.877783 mapread 0x239000 thru 0x2489c4 (0xf9c5 bytes) 000326[1] 1736988745.907755 mapread 0x083000 thru 0x08c444 (0x9445 bytes) 000327[1] 1736988745.928951 mapread 0x6b3000 thru 0x6b42e0 (0x12e1 bytes) 000329[1] 1736988745.933867 mapread 0x700000 thru 0x702b53 (0x2b54 bytes) 000330[1] 1736988745.940594 trunc from 0x83ee14 to 0x0570e7 (0x7e7d2c bytes) 000332[0] 1736988745.951351 read 0x01c000 thru 0x025c54 (0x9c55 bytes) 000333[1] 1736988745.960306 mapwrite 0x839000 thru 0x84097b (0x797c bytes) 000334[0] 1736988745.995221 mapwrite 0x3bb000 thru 0x3bd9a7 (0x29a8 bytes) 000340[1] 1736988746.061063 trunc from 0x84097c to 0x0cfce7 (0x770c94 bytes) 000341[0] 1736988746.073316 trunc from 0x0cfce8 to 0x3e5e88 (0x3161a1 bytes) 000343[1] 1736988746.078711 read_OD 0x137000 thru 0x13cfff (0x6000 bytes) 000344[1] 1736988746.085046 write_OD 0x7fb000 thru 0x801fff (0x7000 bytes) 000345[0] 1736988746.095778 trunc from 0x802000 to 0x280ce5 (0x58131a bytes) 000349[0] 1736988746.108704 trunc from 0x280ce6 to 0x8fb2b7 (0x67a5d2 bytes) 000351[0] 1736988746.123755 write 0x7d3000 thru 0x7e1760 (0xe761 bytes) 000352[0] 1736988746.126873 write 0x6fd000 thru 0x709b10 (0xcb11 bytes) 000357[0] 1736988746.133674 read 0x12a000 thru 0x13678f (0xc790 bytes) 000360[1] 1736988746.176533 read_OD 0x713000 thru 0x71bfff (0x9000 bytes) 000361[1] 1736988746.196803 mapwrite 0x10b000 thru 0x112e0c (0x7e0d bytes) 000362[0] 1736988746.235996 trunc from 0x8fb2b8 to 0x10b4b6 (0x7efe01 bytes) 000364[1] 1736988746.256659 trunc from 0x10b4b7 to 0x8aa0d8 (0x79ec22 bytes) 000365[1] 1736988746.268653 mapwrite 0x9bf000 thru 0x9c505a (0x605b bytes) 000366[0] 1736988746.306949 mapread 0x70d000 thru 0x715101 (0x8102 bytes) 000369[1] 1736988746.346660 write_OD 0x760000 thru 0x76dfff (0xe000 bytes) 000372[1] 1736988746.367945 mapwrite 0x90b000 thru 0x910a0d (0x5a0e bytes) 000373[1] 1736988746.405545 mapwrite 0x3dc000 thru 0x3e569d (0x969e bytes) 000374[1] 1736988746.448810 mapread 0x1ba000 thru 0x1baaa6 (0xaa7 bytes) 000377[0] 1736988746.466914 read 0x344000 thru 0x351aaf (0xdab0 bytes) 000378[1] 1736988746.490812 mapwrite 0x57a000 thru 0x57dd05 (0x3d06 bytes) 000379[1] 1736988746.520318 write 0x5b7000 thru 0x5bb2a1 (0x42a2 bytes) 000380[0] 1736988746.525389 trunc from 0x9c505b to 0x3f9b88 (0x5cb4d2 bytes) 000381[1] 1736988746.532663 mapwrite 0x96d000 thru 0x970db8 (0x3db9 bytes) 000382[1] 1736988746.560918 mapread 0x59d000 thru 0x5a8d52 (0xbd53 bytes) 000383[1] 1736988746.571264 read 0x4c0000 thru 0x4c8f30 (0x8f31 bytes) 000384[1] 1736988746.575155 trunc from 0x970db9 to 0x09b158 (0x8d5c60 bytes) 000385[1] 1736988746.584377 mapwrite 0x04f000 thru 0x05508a (0x608b bytes) 000388[0] 1736988746.604861 trunc from 0x09b159 to 0x110575 (0x7541d bytes) 000395[0] 1736988746.638968 mapwrite 0x433000 thru 0x442c32 (0xfc33 bytes) 000396[0] 1736988746.688834 write 0x155000 thru 0x1623af (0xd3b0 bytes) 000398[0] 1736988746.699772 trunc from 0x442c33 to 0x07bd52 (0x3c6ee0 bytes) 000399[0] 1736988746.705753 trunc from 0x07bd53 to 0x1466c8 (0xca976 bytes) 000400[1] 1736988746.710956 write 0x3dc000 thru 0x3e15b0 (0x55b1 bytes) 000403[1] 1736988746.728643 write 0x1cf000 thru 0x1d7dbc (0x8dbd bytes) 000404[1] 1736988746.737011 read 0x394000 thru 0x39c059 (0x805a bytes) 000405[0] 1736988746.744970 mapwrite 0x711000 thru 0x71c6a4 (0xb6a5 bytes) 000406[1] 1736988746.792657 trunc from 0x71c6a5 to 0x81e24a (0x101ba6 bytes) 000408[0] 1736988746.804316 trunc from 0x81e24b to 0x8c707d (0xa8e33 bytes) 000409[1] 1736988746.817369 write 0x62a000 thru 0x62cf2b (0x2f2c bytes) 000410[1] 1736988746.839078 read 0x21d000 thru 0x222408 (0x5409 bytes) 000411[0] 1736988746.847267 mapwrite 0x05d000 thru 0x0650a0 (0x80a1 bytes) 000412[1] 1736988746.895280 read 0x42d000 thru 0x439e6d (0xce6e bytes) 000414[0] 1736988746.929999 mapread 0x216000 thru 0x21db49 (0x7b4a bytes) 000416[0] 1736988746.944041 write 0x44e000 thru 0x450b4f (0x2b50 bytes) 000418[1] 1736988746.967650 write 0x9fc000 thru 0x9ff726 (0x3727 bytes) truncating to largest ever: 0x9d0dc8 000421[0] 1736988746.985262 trunc from 0x9ff727 to 0x9d0dc7 (0x2e95f bytes) 000424[0] 1736988747.011804 mapread 0x7d1000 thru 0x7d1ae1 (0xae2 bytes) 000425[0] 1736988747.015901 mapwrite 0x903000 thru 0x903ffb (0xffc bytes) 000427[0] 1736988747.043033 mapwrite 0x690000 thru 0x69a52e (0xa52f bytes) 000431[0] 1736988747.094747 write 0x700000 thru 0x703d95 (0x3d96 bytes) 000433[1] 1736988747.109478 mapread 0x537000 thru 0x53f28d (0x828e bytes) 000434[1] 1736988747.143304 mapwrite 0x626000 thru 0x626473 (0x474 bytes) 000436[1] 1736988747.164558 read 0x234000 thru 0x243da3 (0xfda4 bytes) 000437[1] 1736988747.215457 write 0x123000 thru 0x12fcff (0xcd00 bytes) 000438[0] 1736988747.218402 write 0x350000 thru 0x35ce79 (0xce7a bytes) 000439[1] 1736988747.238040 read 0x727000 thru 0x72ea3e (0x7a3f bytes) 000442[0] 1736988747.263340 read 0x579000 thru 0x57ed6b (0x5d6c bytes) 000443[0] 1736988747.291636 trunc from 0x9d0dc8 to 0x9c2c7f (0xe148 bytes) 000445[1] 1736988747.317112 read 0x320000 thru 0x322c9e (0x2c9f bytes) 000447[0] 1736988747.345912 mapread 0x160000 thru 0x16cc4c (0xcc4d bytes) 000449[0] 1736988747.376028 write 0x610000 thru 0x616d26 (0x6d27 bytes) 000450[1] 1736988747.393079 write 0x13d000 thru 0x14b1e9 (0xe1ea bytes) 000453[1] 1736988747.411003 trunc from 0x9c2c80 to 0x2247bc (0x79e4c3 bytes) 000458[0] 1736988747.433737 mapwrite 0x399000 thru 0x39d0a8 (0x40a9 bytes) 000459[1] 1736988747.471195 write 0x916000 thru 0x91f392 (0x9393 bytes) 000460[0] 1736988747.475220 write 0x12d000 thru 0x13921c (0xc21d bytes) 000462[0] 1736988747.488569 trunc from 0x91f393 to 0x2e05a6 (0x63edec bytes) 000463[0] 1736988747.494480 mapread 0x0a7000 thru 0x0b0558 (0x9559 bytes) 000465[1] 1736988747.516300 read 0x19f000 thru 0x1aa55c (0xb55d bytes) 000466[1] 1736988747.532534 mapread 0x13a000 thru 0x149ca5 (0xfca6 bytes) 000467[1] 1736988747.556946 write 0x48d000 thru 0x4921e1 (0x51e2 bytes) 000468[0] 1736988747.562267 read 0x3b9000 thru 0x3c23f6 (0x93f7 bytes) 000470[1] 1736988747.577058 mapread 0x179000 thru 0x181c97 (0x8c98 bytes) 000471[0] 1736988747.603776 mapwrite 0x0d6000 thru 0x0e404b (0xe04c bytes) 000472[0] 1736988747.666286 mapread 0x25d000 thru 0x260ed2 (0x3ed3 bytes) 000475[0] 1736988747.692777 write 0x254000 thru 0x259def (0x5df0 bytes) 000476[0] 1736988747.701370 mapread 0x32b000 thru 0x330728 (0x5729 bytes) 000478[0] 1736988747.720423 trunc from 0x4921e2 to 0x8604e8 (0x3ce307 bytes) 000480[1] 1736988747.738657 trunc from 0x8604e9 to 0x70d5a5 (0x152f43 bytes) 000481[0] 1736988747.759769 mapread 0x282000 thru 0x28d7c0 (0xb7c1 bytes) 000483[1] 1736988747.812427 mapwrite 0x35b000 thru 0x3657d2 (0xa7d3 bytes) 000484[1] 1736988747.877691 trunc from 0x70d5a6 to 0x88c10e (0x17eb69 bytes) 000485[1] 1736988747.885825 trunc from 0x88c10f to 0x30daa5 (0x57e669 bytes) 000486[0] 1736988747.901281 read 0x0ab000 thru 0x0b5c8c (0xac8d bytes) 000492[0] 1736988747.926315 read 0x08f000 thru 0x0967b6 (0x77b7 bytes) 000493[0] 1736988747.930086 write 0x604000 thru 0x6052c6 (0x12c7 bytes) 000494[1] 1736988747.938031 write 0x454000 thru 0x45b551 (0x7552 bytes) 000495[1] 1736988747.950475 mapread 0x2e4000 thru 0x2eee46 (0xae47 bytes) 000498[0] 1736988747.975169 read 0x3f8000 thru 0x3fcd80 (0x4d81 bytes) 000500[0] 1736988747.993867 trunc from 0x6052c7 to 0x3d9e09 (0x22b4bd bytes) All operations completed A-OK! PASS 16a (16s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 19:52:29 (1736988749) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0772269 s, 27.2 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0543525 s, 38.6 MB/s Chance of close/open is 1 in 50 Seed set to 8749 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read skipping zero size read 000007[0] 1736988750.038031 mapwrite 0x4d48a9 thru 0x4d535c (0xab4 bytes) 000009[1] 1736988750.076261 write 0x8b7723 thru 0x8b82b2 (0xb90 bytes) 000010[1] 1736988750.085313 read 0x5eea42 thru 0x5ef75f (0xd1e bytes) 000011[0] 1736988750.089176 mapwrite 0x585257 thru 0x587015 (0x1dbf bytes) truncating to largest ever: 0x9fee8c 000012[1] 1736988750.135775 trunc from 0x8b82b3 to 0x9fee8b (0x146bd9 bytes) 000013[0] 1736988750.150341 write 0x4f29ce thru 0x4f3c44 (0x1277 bytes) 000014[0] 1736988750.160130 read 0x8ef121 thru 0x8f0ee7 (0x1dc7 bytes) 000019[0] 1736988750.185909 read 0x1af726 thru 0x1b169e (0x1f79 bytes) 000021[1] 1736988750.209377 read 0x52fe1e thru 0x530186 (0x369 bytes) 000024[1] 1736988750.235678 mapread 0x7c85d7 thru 0x7c86ae (0xd8 bytes) 000025[1] 1736988750.245834 mapwrite 0x342ce0 thru 0x343a98 (0xdb9 bytes) 000026[1] 1736988750.282739 trunc from 0x9fee8c to 0x1aa336 (0x854b55 bytes) 000028[1] 1736988750.305790 mapwrite 0x58cdb6 thru 0x58ed74 (0x1fbf bytes) 000029[1] 1736988750.348318 read 0x4e9f2f thru 0x4eb19a (0x126c bytes) 000034[1] 1736988750.368991 read 0x4398c6 thru 0x43a265 (0x9a0 bytes) 000035[1] 1736988750.380548 read 0x473526 thru 0x473f97 (0xa72 bytes) 000036[1] 1736988750.380943 mapwrite 0x7f4e2a thru 0x7f5e90 (0x1067 bytes) 000037[1] 1736988750.407469 mapwrite 0x082a88 thru 0x083657 (0xbd0 bytes) 000038[0] 1736988750.426087 read 0x14ffa4 thru 0x1506fd (0x75a bytes) 000040[1] 1736988750.443776 trunc from 0x7f5e91 to 0x6166bd (0x1df7d3 bytes) 000041[0] 1736988750.450950 trunc from 0x6166be to 0x392bb4 (0x283b09 bytes) 000042[1] 1736988750.463659 read 0x11f0da thru 0x11fdc3 (0xcea bytes) 000044[1] 1736988750.478491 mapread 0x3101ed thru 0x310a94 (0x8a8 bytes) 000045[1] 1736988750.484602 mapread 0x18cfc9 thru 0x18e7a6 (0x17de bytes) 000047[0] 1736988750.492552 mapread 0x36d3cc thru 0x36d849 (0x47e bytes) 000048[0] 1736988750.501548 mapwrite 0x383db9 thru 0x3853e9 (0x1631 bytes) 000049[1] 1736988750.537662 mapread 0x179e79 thru 0x17a5f8 (0x780 bytes) 000052[0] 1736988750.560115 mapwrite 0x276c92 thru 0x278b2e (0x1e9d bytes) 000053[0] 1736988750.578114 read 0x03fa90 thru 0x0412c6 (0x1837 bytes) 000055[1] 1736988750.602792 write 0x1b75ef thru 0x1b7c5a (0x66c bytes) 000064[1] 1736988750.666966 trunc from 0x392bb5 to 0x47d7fa (0xeac46 bytes) 000066[1] 1736988750.693996 write 0x9bc71f thru 0x9bd325 (0xc07 bytes) 000067[1] 1736988750.702875 trunc from 0x9bd326 to 0x6a61b8 (0x31716d bytes) 000068[0] 1736988750.710486 write 0x4e65da thru 0x4e7ab8 (0x14df bytes) 000070[0] 1736988750.733604 trunc from 0x6a61b9 to 0x910216 (0x26a05e bytes) 000073[1] 1736988750.746137 read 0x6b9c0b thru 0x6bb2a6 (0x169c bytes) 000075[0] 1736988750.765926 write 0x937699 thru 0x9395d9 (0x1f41 bytes) 000077[0] 1736988750.767504 mapread 0x191734 thru 0x1918a2 (0x16f bytes) 000078[1] 1736988750.788092 trunc from 0x9395da to 0x066cfc (0x8d28dd bytes) 000079[0] 1736988750.807003 mapwrite 0x96d9b6 thru 0x96e4b9 (0xb04 bytes) 000083[0] 1736988750.846107 read 0x089068 thru 0x089b76 (0xb0f bytes) 000085[1] 1736988750.898152 mapread 0x410098 thru 0x411fb4 (0x1f1d bytes) 000086[0] 1736988750.923320 read 0x6f51f6 thru 0x6f5874 (0x67f bytes) 000089[0] 1736988750.953093 write 0x562c95 thru 0x563609 (0x975 bytes) 000092[0] 1736988750.960949 trunc from 0x96e4ba to 0x29be90 (0x6d2629 bytes) 000093[0] 1736988750.979901 write 0x198e5f thru 0x19aafc (0x1c9e bytes) 000094[0] 1736988750.980755 write 0x1e8fc2 thru 0x1ea951 (0x1990 bytes) 000095[0] 1736988750.981606 mapread 0x204450 thru 0x205e22 (0x19d3 bytes) 000096[1] 1736988750.993787 mapwrite 0x050626 thru 0x050ca7 (0x682 bytes) 000099[0] 1736988751.034993 trunc from 0x29be91 to 0x078709 (0x223787 bytes) 000101[1] 1736988751.045005 read 0x00f83e thru 0x011118 (0x18db bytes) 000102[0] 1736988751.057650 read 0x03e207 thru 0x040064 (0x1e5e bytes) 000103[0] 1736988751.063617 trunc from 0x07870a to 0x0bd9c5 (0x452bc bytes) 000104[0] 1736988751.077540 write 0x9dd8fa thru 0x9df809 (0x1f10 bytes) 000107[0] 1736988751.097165 mapwrite 0x12134b thru 0x122bda (0x1890 bytes) 000112[0] 1736988751.141295 mapread 0x5ec56b thru 0x5ec916 (0x3ac bytes) 000113[0] 1736988751.141630 trunc from 0x9df80a to 0x1007fd (0x8df00c bytes) 000114[1] 1736988751.153925 read 0x0ace6a thru 0x0adf51 (0x10e8 bytes) 000115[1] 1736988751.162673 mapread 0x0d7f8d thru 0x0d9a07 (0x1a7b bytes) 000116[1] 1736988751.165484 write 0x0a400b thru 0x0a44fe (0x4f4 bytes) 000119[0] 1736988751.183912 write 0x26bfc5 thru 0x26c5ea (0x626 bytes) 000120[0] 1736988751.188589 trunc from 0x26c5eb to 0x2de3ee (0x71e04 bytes) 000125[1] 1736988751.204496 mapwrite 0x238b46 thru 0x239136 (0x5f1 bytes) 000127[1] 1736988751.246549 trunc from 0x2de3ef to 0x25b88e (0x82b60 bytes) 000130[0] 1736988751.256222 trunc from 0x25b88f to 0x466605 (0x20ad77 bytes) 000131[1] 1736988751.273358 read 0x2aecb4 thru 0x2af693 (0x9e0 bytes) 000133[1] 1736988751.290818 read 0x1dea50 thru 0x1dfe60 (0x1411 bytes) 000135[1] 1736988751.308744 mapread 0x2d00a3 thru 0x2d0cfd (0xc5b bytes) 000137[1] 1736988751.309274 trunc from 0x466606 to 0x3eb8ab (0x7ad5a bytes) 000139[0] 1736988751.331287 read 0x0eac18 thru 0x0ebea1 (0x128a bytes) 000140[1] 1736988751.339918 trunc from 0x3eb8ac to 0x6ed625 (0x301d7a bytes) 000141[0] 1736988751.352299 read 0x5fe0f2 thru 0x5ffbdc (0x1aeb bytes) 000143[0] 1736988751.365025 read 0x1685be thru 0x16918b (0xbce bytes) 000149[1] 1736988751.396912 read 0x5c2278 thru 0x5c4084 (0x1e0d bytes) 000150[1] 1736988751.403408 mapwrite 0x0572fa thru 0x05765c (0x363 bytes) 000152[1] 1736988751.418086 write 0x9b1b60 thru 0x9b3734 (0x1bd5 bytes) 000153[0] 1736988751.421692 mapread 0x9930d8 thru 0x994eb4 (0x1ddd bytes) 000155[0] 1736988751.439280 mapread 0x573cd1 thru 0x57418b (0x4bb bytes) 000156[1] 1736988751.448596 mapread 0x77164e thru 0x771940 (0x2f3 bytes) 000157[0] 1736988751.466623 trunc from 0x9b3735 to 0x8f7b07 (0xbbc2d bytes) 000160[0] 1736988751.486236 mapread 0x2f4d59 thru 0x2f6092 (0x133a bytes) 000161[1] 1736988751.517754 mapread 0x37c1b1 thru 0x37dbcb (0x1a1b bytes) 000165[0] 1736988751.542874 read 0x2e5072 thru 0x2e6b18 (0x1aa7 bytes) 000166[0] 1736988751.543152 mapwrite 0x5269f5 thru 0x526f11 (0x51d bytes) 000167[0] 1736988751.576562 trunc from 0x8f7b08 to 0x008719 (0x8ef3ee bytes) 000169[0] 1736988751.592661 mapwrite 0x45e5c8 thru 0x45f86f (0x12a8 bytes) 000171[0] 1736988751.615026 trunc from 0x45f870 to 0x38e39d (0xd14d2 bytes) 000172[1] 1736988751.622536 mapread 0x0b7c74 thru 0x0b8256 (0x5e3 bytes) 000176[1] 1736988751.640277 mapwrite 0x840552 thru 0x840e97 (0x946 bytes) 000177[1] 1736988751.668869 mapwrite 0x4e033a thru 0x4e163c (0x1303 bytes) 000179[0] 1736988751.687904 mapread 0x1ab998 thru 0x1ac082 (0x6eb bytes) 000180[0] 1736988751.695828 mapwrite 0x6f8607 thru 0x6f9863 (0x125d bytes) 000181[0] 1736988751.716874 read 0x38f186 thru 0x38fbef (0xa6a bytes) 000182[0] 1736988751.739448 write 0x5ca8a0 thru 0x5cc23b (0x199c bytes) 000183[1] 1736988751.746707 mapwrite 0x83379c thru 0x835180 (0x19e5 bytes) 000190[0] 1736988751.801878 trunc from 0x840e98 to 0x3068ee (0x53a5a9 bytes) 000192[0] 1736988751.816005 mapwrite 0x241585 thru 0x24160f (0x8b bytes) 000195[1] 1736988751.844224 trunc from 0x3068ef to 0x1f363e (0x1132b0 bytes) 000196[0] 1736988751.857729 read 0x06d191 thru 0x06df78 (0xde8 bytes) 000197[0] 1736988751.870530 trunc from 0x1f363f to 0x66c29a (0x478c5c bytes) 000199[1] 1736988751.884963 trunc from 0x66c29b to 0x716b9b (0xaa901 bytes) 000200[0] 1736988751.893833 write 0x45ff80 thru 0x460998 (0xa19 bytes) 000201[1] 1736988751.903812 mapread 0x555880 thru 0x555a1b (0x19c bytes) 000202[0] 1736988751.912372 read 0x63bdc1 thru 0x63bed4 (0x114 bytes) 000205[1] 1736988751.938127 trunc from 0x716b9c to 0x89974d (0x182bb2 bytes) 000207[1] 1736988751.956654 write 0x9a52a7 thru 0x9a7108 (0x1e62 bytes) 000208[1] 1736988751.965636 trunc from 0x9a7109 to 0x4a8120 (0x4fefe8 bytes) 000209[0] 1736988751.972740 write 0x0646df thru 0x06486d (0x18f bytes) 000210[1] 1736988751.983390 mapread 0x2b4e3f thru 0x2b60f6 (0x12b8 bytes) 000211[1] 1736988751.995266 trunc from 0x4a8121 to 0x2587eb (0x24f935 bytes) 000213[1] 1736988752.015330 mapread 0x002615 thru 0x002e86 (0x872 bytes) 000214[1] 1736988752.022885 write 0x9b463d thru 0x9b6011 (0x19d5 bytes) 000215[1] 1736988752.026654 mapread 0x4f4b97 thru 0x4f6746 (0x1bb0 bytes) 000216[1] 1736988752.036790 mapwrite 0x68fa01 thru 0x69156b (0x1b6b bytes) 000217[1] 1736988752.064170 mapread 0x660973 thru 0x66165c (0xcea bytes) 000218[0] 1736988752.067309 read 0x61b0ad thru 0x61c658 (0x15ac bytes) 000219[0] 1736988752.075370 mapwrite 0x5b0f91 thru 0x5b11fb (0x26b bytes) 000225[0] 1736988752.109771 trunc from 0x9b6012 to 0x18bc0d (0x82a404 bytes) 000226[0] 1736988752.122444 trunc from 0x18bc0e to 0x10fe84 (0x7bd89 bytes) 000227[0] 1736988752.129675 mapread 0x09320b thru 0x09424a (0x1040 bytes) 000229[0] 1736988752.138613 trunc from 0x10fe85 to 0x3edde6 (0x2ddf62 bytes) 000231[0] 1736988752.147410 trunc from 0x3edde7 to 0x880b69 (0x492d83 bytes) 000233[0] 1736988752.156843 read 0x52b2a7 thru 0x52b403 (0x15d bytes) 000235[0] 1736988752.174852 read 0x246cdf thru 0x246e86 (0x1a8 bytes) 000236[0] 1736988752.185792 mapread 0x446935 thru 0x4482c6 (0x1992 bytes) 000239[0] 1736988752.190216 read 0x43e8e2 thru 0x440421 (0x1b40 bytes) 000241[0] 1736988752.198416 mapwrite 0x803a6c thru 0x804280 (0x815 bytes) 000243[1] 1736988752.217996 trunc from 0x880b6a to 0x463dbe (0x41cdab bytes) 000246[0] 1736988752.239928 write 0x8652da thru 0x867214 (0x1f3b bytes) 000247[0] 1736988752.245780 mapread 0x64e366 thru 0x6501b9 (0x1e54 bytes) 000251[0] 1736988752.264026 read 0x39ec83 thru 0x39f977 (0xcf5 bytes) 000254[0] 1736988752.284126 read 0x2a19ec thru 0x2a344c (0x1a61 bytes) 000255[0] 1736988752.292652 write 0x3415b8 thru 0x341dcf (0x818 bytes) 000256[0] 1736988752.302156 read 0x6b9b22 thru 0x6bb781 (0x1c60 bytes) 000259[0] 1736988752.309285 write 0x19f26a thru 0x19fac9 (0x860 bytes) 000260[0] 1736988752.310984 trunc from 0x867215 to 0x794060 (0xd31b4 bytes) 000262[1] 1736988752.322440 read 0x1fe88b thru 0x1ff8e1 (0x1057 bytes) 000263[1] 1736988752.333076 write 0x8152d8 thru 0x815fb2 (0xcdb bytes) 000264[1] 1736988752.339420 write 0x3f71fa thru 0x3f875e (0x1565 bytes) 000265[0] 1736988752.359170 mapwrite 0x1ae304 thru 0x1ae4e8 (0x1e5 bytes) 000266[1] 1736988752.393372 mapread 0x1922b3 thru 0x1933e5 (0x1133 bytes) 000267[1] 1736988752.401160 trunc from 0x815fb3 to 0x5d58a4 (0x24070e bytes) 000268[1] 1736988752.409490 write 0x8769f7 thru 0x876c9e (0x2a8 bytes) 000270[1] 1736988752.416818 mapwrite 0x8fb20a thru 0x8fc9da (0x17d1 bytes) 000272[1] 1736988752.459371 mapwrite 0x5c1af0 thru 0x5c32f4 (0x1805 bytes) 000273[0] 1736988752.479460 trunc from 0x8fc9db to 0x37c462 (0x580578 bytes) 000274[1] 1736988752.496068 write 0x2148c3 thru 0x216689 (0x1dc7 bytes) 000275[1] 1736988752.505905 mapwrite 0x15c865 thru 0x15e7b0 (0x1f4c bytes) 000276[1] 1736988752.537016 mapwrite 0x6b9286 thru 0x6bb0e2 (0x1e5d bytes) 000279[0] 1736988752.576487 read 0x25243b thru 0x253231 (0xdf7 bytes) 000280[0] 1736988752.589224 mapread 0x0e9e22 thru 0x0ea163 (0x342 bytes) 000283[0] 1736988752.605850 mapwrite 0x826e61 thru 0x828999 (0x1b39 bytes) 000284[0] 1736988752.670450 trunc from 0x82899a to 0x3cb9e0 (0x45cfb9 bytes) 000289[0] 1736988752.692154 mapread 0x1d16d9 thru 0x1d2c2f (0x1557 bytes) 000291[1] 1736988752.698908 mapread 0x0ae9cc thru 0x0affd6 (0x160b bytes) 000293[0] 1736988752.713076 mapread 0x2b1f25 thru 0x2b2df3 (0xecf bytes) 000294[1] 1736988752.722739 trunc from 0x3cb9e1 to 0x077681 (0x35435f bytes) 000295[1] 1736988752.736411 mapwrite 0x97511f thru 0x975810 (0x6f2 bytes) 000298[1] 1736988752.776325 read 0x7458e4 thru 0x7474fa (0x1c17 bytes) 000302[1] 1736988752.807171 read 0x830640 thru 0x831f1b (0x18dc bytes) 000303[1] 1736988752.814164 mapwrite 0x4e5f3c thru 0x4e62be (0x383 bytes) 000306[0] 1736988752.841494 read 0x166b42 thru 0x1671a4 (0x663 bytes) 000308[1] 1736988752.870764 trunc from 0x975811 to 0x379ee9 (0x5fb927 bytes) 000312[1] 1736988752.910552 mapwrite 0x594d1f thru 0x594d81 (0x63 bytes) 000313[0] 1736988752.949021 read 0x0478ab thru 0x04875f (0xeb5 bytes) 000316[0] 1736988752.984280 read 0x1dc4e8 thru 0x1de49b (0x1fb4 bytes) 000317[1] 1736988753.013106 trunc from 0x594d82 to 0x3d48d3 (0x1c04ae bytes) 000318[1] 1736988753.041445 mapread 0x24e255 thru 0x24e332 (0xde bytes) 000319[0] 1736988753.046309 trunc from 0x3d48d4 to 0x30394e (0xd0f85 bytes) 000321[1] 1736988753.067140 mapread 0x1d4d52 thru 0x1d62a1 (0x1550 bytes) 000324[1] 1736988753.090659 read 0x015524 thru 0x0160ec (0xbc9 bytes) 000326[0] 1736988753.110835 trunc from 0x30394f to 0x75cf91 (0x459643 bytes) 000328[0] 1736988753.128120 write 0x3e8678 thru 0x3e9aa4 (0x142d bytes) 000334[0] 1736988753.149246 trunc from 0x75cf92 to 0x408c06 (0x35438b bytes) 000336[0] 1736988753.160438 mapread 0x3e6334 thru 0x3e7836 (0x1503 bytes) 000339[1] 1736988753.169685 write 0x93c4fd thru 0x93d3e3 (0xee7 bytes) 000340[1] 1736988753.177819 mapwrite 0x1ce1f7 thru 0x1cec5d (0xa67 bytes) 000342[0] 1736988753.211799 mapwrite 0x317f60 thru 0x3189cf (0xa70 bytes) 000343[1] 1736988753.246301 write 0x3d2ed5 thru 0x3d4152 (0x127e bytes) 000344[1] 1736988753.255551 write 0x4d890b thru 0x4d9558 (0xc4e bytes) 000345[0] 1736988753.272680 read 0x88699c thru 0x8883c2 (0x1a27 bytes) 000346[0] 1736988753.288686 write 0x0c3a8e thru 0x0c494d (0xec0 bytes) 000347[1] 1736988753.297278 mapwrite 0x9c9608 thru 0x9ca797 (0x1190 bytes) 000348[1] 1736988753.328563 mapwrite 0x3351b9 thru 0x33634f (0x1197 bytes) 000349[0] 1736988753.348675 mapwrite 0x972a4a thru 0x974178 (0x172f bytes) 000355[1] 1736988753.397764 mapwrite 0x2a03c1 thru 0x2a0a90 (0x6d0 bytes) 000356[0] 1736988753.417348 write 0x268026 thru 0x268465 (0x440 bytes) 000357[1] 1736988753.430326 read 0x297ae2 thru 0x298cd9 (0x11f8 bytes) 000359[0] 1736988753.440000 write 0x97ba82 thru 0x97d7c5 (0x1d44 bytes) 000361[1] 1736988753.448272 trunc from 0x9ca798 to 0x7ddea6 (0x1ec8f1 bytes) 000363[0] 1736988753.466322 write 0x2c2911 thru 0x2c46e0 (0x1dd0 bytes) 000364[1] 1736988753.473634 mapread 0x6eb721 thru 0x6eca85 (0x1365 bytes) 000366[1] 1736988753.506711 trunc from 0x7ddea7 to 0x38d8c2 (0x4505e4 bytes) 000367[0] 1736988753.521389 trunc from 0x38d8c3 to 0x3948bf (0x6ffd bytes) 000369[0] 1736988753.529323 mapwrite 0x5bec1c thru 0x5bed91 (0x176 bytes) 000374[0] 1736988753.563723 trunc from 0x5bed92 to 0x57365a (0x4b737 bytes) 000377[0] 1736988753.581889 trunc from 0x57365b to 0x07f8fd (0x4f3d5d bytes) 000380[1] 1736988753.593900 trunc from 0x07f8fe to 0x58ee89 (0x50f58c bytes) 000381[0] 1736988753.602943 write 0x7edee5 thru 0x7ef356 (0x1472 bytes) 000382[1] 1736988753.609859 write 0x44afc1 thru 0x44b532 (0x572 bytes) 000383[0] 1736988753.617219 write 0x9b53f8 thru 0x9b6548 (0x1151 bytes) 000384[1] 1736988753.621751 mapwrite 0x87d7ab thru 0x87f085 (0x18db bytes) 000386[0] 1736988753.669478 write 0x3682a6 thru 0x3699d2 (0x172d bytes) 000390[0] 1736988753.689284 read 0x2a8210 thru 0x2aa027 (0x1e18 bytes) 000391[0] 1736988753.717126 mapwrite 0x1f5719 thru 0x1f59da (0x2c2 bytes) 000392[1] 1736988753.745991 write 0x982e0e thru 0x983498 (0x68b bytes) 000393[1] 1736988753.759265 mapwrite 0x7e2a4a thru 0x7e39af (0xf66 bytes) 000394[0] 1736988753.778272 write 0x60d36c thru 0x60d8d1 (0x566 bytes) 000395[1] 1736988753.790437 mapwrite 0x36be99 thru 0x36c6a0 (0x808 bytes) 000396[0] 1736988753.812533 mapread 0x8c7fce thru 0x8c8798 (0x7cb bytes) 000399[0] 1736988753.835654 mapwrite 0x48e1de thru 0x48f8b3 (0x16d6 bytes) 000401[1] 1736988753.862318 write 0x284a37 thru 0x286991 (0x1f5b bytes) 000402[1] 1736988753.867402 mapwrite 0x6a4459 thru 0x6a4ab5 (0x65d bytes) 000405[1] 1736988753.889117 mapwrite 0x1375c4 thru 0x137d3a (0x777 bytes) 000406[1] 1736988753.905196 trunc from 0x9b6549 to 0x63048c (0x3860bc bytes) 000407[0] 1736988753.911239 trunc from 0x63048d to 0x4529b8 (0x1ddad4 bytes) 000409[0] 1736988753.922280 mapread 0x34bebb thru 0x34d59b (0x16e1 bytes) 000410[0] 1736988753.927138 mapwrite 0x247212 thru 0x247857 (0x646 bytes) 000416[1] 1736988753.965712 mapwrite 0x14d313 thru 0x14d61b (0x309 bytes) 000418[1] 1736988754.002003 trunc from 0x4529b9 to 0x6d55a0 (0x282be8 bytes) 000419[1] 1736988754.011498 read 0x62740d thru 0x6275a5 (0x199 bytes) 000420[1] 1736988754.020129 write 0x8e9e4a thru 0x8ebc64 (0x1e1b bytes) 000422[0] 1736988754.028161 trunc from 0x8ebc65 to 0x9812b4 (0x95650 bytes) 000426[0] 1736988754.055958 trunc from 0x9812b5 to 0x7a04c7 (0x1e0ded bytes) 000428[0] 1736988754.069556 trunc from 0x7a04c8 to 0x798397 (0x8130 bytes) 000430[1] 1736988754.076040 mapread 0x36eb32 thru 0x36fd19 (0x11e8 bytes) 000431[1] 1736988754.086455 write 0x935892 thru 0x937096 (0x1805 bytes) 000432[1] 1736988754.095587 mapread 0x6655d0 thru 0x6669cd (0x13fe bytes) 000433[0] 1736988754.115227 trunc from 0x937097 to 0x884e98 (0xb21fe bytes) 000434[1] 1736988754.134065 write 0x9a4883 thru 0x9a599d (0x111b bytes) 000435[0] 1736988754.141216 mapwrite 0x4209bb thru 0x42116c (0x7b2 bytes) 000436[1] 1736988754.163830 trunc from 0x9a599e to 0x8eecd9 (0xb6cc4 bytes) 000439[0] 1736988754.169320 mapread 0x8d9c0c thru 0x8daa37 (0xe2c bytes) 000442[0] 1736988754.180383 read 0x8e87f6 thru 0x8e88c8 (0xd3 bytes) 000445[0] 1736988754.188796 mapwrite 0x7078c7 thru 0x709661 (0x1d9b bytes) 000447[1] 1736988754.209560 read 0x1f5047 thru 0x1f5e80 (0xe3a bytes) 000448[0] 1736988754.235804 mapwrite 0x7ec081 thru 0x7ecbaa (0xb2a bytes) 000450[0] 1736988754.261540 mapwrite 0x75c0df thru 0x75c991 (0x8b3 bytes) 000451[1] 1736988754.274985 read 0x688206 thru 0x688bc6 (0x9c1 bytes) 000452[1] 1736988754.275223 write 0x8beeb7 thru 0x8c07b8 (0x1902 bytes) 000453[0] 1736988754.290728 trunc from 0x8eecda to 0x5af6ab (0x33f62e bytes) 000454[1] 1736988754.305796 write 0x7fe190 thru 0x7ff454 (0x12c5 bytes) 000455[0] 1736988754.316824 read 0x2c2b6b thru 0x2c2e8b (0x321 bytes) 000456[0] 1736988754.326061 mapwrite 0x4fffc9 thru 0x5018ec (0x1924 bytes) 000458[1] 1736988754.356421 trunc from 0x7ff455 to 0x6d4ff1 (0x12a463 bytes) 000460[1] 1736988754.367740 read 0x34bbe5 thru 0x34c4b6 (0x8d2 bytes) 000462[0] 1736988754.385917 mapread 0x5bdadc thru 0x5be874 (0xd99 bytes) 000463[1] 1736988754.394440 mapread 0x2364da thru 0x236a25 (0x54c bytes) 000464[1] 1736988754.395725 trunc from 0x6d4ff2 to 0x4a57f8 (0x22f7f9 bytes) 000466[1] 1736988754.406092 trunc from 0x4a57f9 to 0x645255 (0x19fa5d bytes) 000468[1] 1736988754.415931 trunc from 0x645256 to 0x0fe21b (0x54703a bytes) 000470[0] 1736988754.425392 write 0x01b59f thru 0x01cd1b (0x177d bytes) 000472[0] 1736988754.449574 write 0x894b5e thru 0x89587e (0xd21 bytes) 000477[1] 1736988754.485379 trunc from 0x89587f to 0x882579 (0x13305 bytes) 000479[1] 1736988754.509485 write 0x3ebf09 thru 0x3ed568 (0x1660 bytes) 000481[0] 1736988754.520863 mapread 0x77ba0b thru 0x77d4b2 (0x1aa8 bytes) 000483[1] 1736988754.540210 mapread 0x6e3e1c thru 0x6e51dd (0x13c2 bytes) 000485[0] 1736988754.562338 write 0x2f21ba thru 0x2f3ffd (0x1e44 bytes) 000486[1] 1736988754.571057 read 0x4f8f99 thru 0x4f96a9 (0x711 bytes) 000487[0] 1736988754.579420 mapwrite 0x71357d thru 0x714400 (0xe84 bytes) 000489[1] 1736988754.620268 mapwrite 0x52dcc9 thru 0x52f380 (0x16b8 bytes) 000490[1] 1736988754.644151 trunc from 0x88257a to 0x933c48 (0xb16cf bytes) 000493[0] 1736988754.659221 mapread 0x1fb268 thru 0x1fcbc8 (0x1961 bytes) 000495[1] 1736988754.678953 mapwrite 0x7f32d2 thru 0x7f3c12 (0x941 bytes) 000496[0] 1736988754.710135 trunc from 0x933c49 to 0x69348d (0x2a07bb bytes) 000497[0] 1736988754.724198 read 0x21075e thru 0x210f4c (0x7ef bytes) 000499[0] 1736988754.767362 write 0x4c4308 thru 0x4c5eb5 (0x1bae bytes) 000500[0] 1736988754.768077 read 0x47f8e5 thru 0x480042 (0x75e bytes) All operations completed A-OK! PASS 16b (7s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 19:52:36 (1736988756) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (1s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 19:52:37 (1736988757) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.12693 s, 16.5 MB/s 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0700277 s, 29.9 MB/s ldlm.namespaces.MGC192.168.201.109@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800ab31c000.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b3d07800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800ab31c000.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800ab31c000.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b3d07800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.18597 s, 25.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.12434 s, 93.3 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.617621 s, 170 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.76403 s, 59.4 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.51468 s, 41.7 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.968732 s, 108 MB/s PASS 16d (15s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 19:52:52 (1736988772) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0727221 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319676 s, 32.8 MB/s PASS 16e (3s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 19:52:55 (1736988775) TIMEOUT PASS 16f (22s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 19:53:17 (1736988797) TIMEOUT PASS 16g (22s) == sanityn test 16i: read after truncate file ============ 19:53:39 (1736988819) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (1s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 19:53:41 (1736988821) fail_loc=0x8000030a PASS 17 (3s) == sanityn test 18: mmap sanity check =========================================================================================== 19:53:44 (1736988824) mmap test1: basic mmap operation (PASS, 0.012822s) mmap test2: MAP_PRIVATE not write back (PASS, 0.010366s) mmap test3: concurrent mmap ops on two nodes (PASS, 2.3005s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 2.0681s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.025974s) mmap test6: check mmap write/read content on two nodes (PASS, 0.0362s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0178s) PASS 18 (9s) == sanityn test 19: test concurrent uncached read races ========================================================================= 19:53:53 (1736988833) oleg109-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg109-client: oleg109-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (1s) == sanityn test 20: test extra readahead page left in cache ============================================================== 19:53:54 (1736988834) PASS 20 (2s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 19:53:56 (1736988836) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (2s) == sanityn test 23: others should see updated atime while another read============================================================== 19:53:58 (1736988838) now is 1736988839 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.6833 new atime is 1736988900 PASS 23 (64s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 19:55:02 (1736988902) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 115.2K 376 114.9K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 376 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 7.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 6.0M 3.6G 1% /mnt/lustre[OST:1] filesystem_summary: 7.2G 13.0M 7.2G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118008 376 117632 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235665 369 235296 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 488.6K 369 488.3K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 115.2K 376 114.9K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 115.3K 373 114.9K 1% /mnt/lustre2[OST:1] filesystem_summary: 230.1K 369 229.8K 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 118008 376 117632 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre[OST:1] filesystem_summary: 235665 369 235296 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 500339 369 499970 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 118008 376 117632 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 118037 373 117664 1% /mnt/lustre2[OST:1] filesystem_summary: 235665 369 235296 1% /mnt/lustre2 No device found for name osc: Invalid argument UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 7168 3762176 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 6144 3763200 1% /mnt/lustre2[OST:1] filesystem_summary: 7542784 13312 7525376 1% /mnt/lustre2 PASS 24a (2s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 19:55:04 (1736988904) PASS 24b (2s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 19:55:06 (1736988906) running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (3s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 19:55:09 (1736988909) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (1s) == sanityn test 26a: allow mtime to get older ============ 19:55:10 (1736988910) utime: good mknod times 1736988909 <= 1736988910 <= 1736988910 for /mnt/lustre/f26a utime: good mknod times 1736988909 <= 1736988910 <= 1736988910 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (3s) == sanityn test 26b: sync mtime between ost and mds ====== 19:55:13 (1736988913) PASS 26b (4s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 19:55:18 (1736988918) 0 PASS 30 (4s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 19:55:22 (1736988922) fail_loc=0x314 PASS 31a (3s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 19:55:25 (1736988925) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg109-server mds-ost sync done. sleep 5 for ZFS zfs Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.201.109@tcp:/lustre 7542784 14336 7524352 1% /mnt/lustre2 PASS 31b (12s) == sanityn test 31r: open-rename(replace) race =========== 19:55:37 (1736988937) -rw-r--r-- 1 root root 0 Jan 15 19:55 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 mv: failed to access '/mnt/lustre2/f31r.sanityn.target': Input/output error PASS 31r (4s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 19:55:42 (1736988942) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (1s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-Lock-Cancel ========================================================== 19:55:43 (1736988943) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (0s) == sanityn test 33d: DNE distributed operation should trigger COS ========================================================== 19:55:43 (1736988943) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (1s) == sanityn test 33e: DNE local operation shouldn't trigger COS ========================================================== 19:55:44 (1736988944) SKIP: sanityn test_33e Need two or more clients, have 1 SKIP 33e (1s) == sanityn test 34: no lock timeout under IO ============= 19:55:45 (1736988945) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg109-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800ab31c000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b3d07800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800ab31c000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0000-osc-ffff8800b3d07800.ost_server_uuid in IDLE state after 0 sec oleg109-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800ab31c000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b3d07800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800ab31c000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800b3d07800.ost_server_uuid in FULL state after 0 sec oleg109-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800ab31c000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b3d07800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800ab31c000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8800b3d07800.ost_server_uuid in IDLE state after 0 sec oleg109-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800ab31c000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b3d07800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800ab31c000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800b3d07800.ost_server_uuid in IDLE state after 0 sec oleg109-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800ab31c000.ost_server_uuid,osc.lustre-OST0000-osc-ffff8800b3d07800.ost_server_uuid 40 osc.lustre-OST0000-osc-ffff8800ab31c000.ost_server_uuid in IDLE state after 0 sec osc.lustre-OST0000-osc-ffff8800b3d07800.ost_server_uuid in IDLE state after 0 sec oleg109-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800ab31c000.ost_server_uuid,osc.lustre-OST0001-osc-ffff8800b3d07800.ost_server_uuid 40 osc.lustre-OST0001-osc-ffff8800ab31c000.ost_server_uuid in FULL state after 0 sec osc.lustre-OST0001-osc-ffff8800b3d07800.ost_server_uuid in IDLE state after 0 sec PASS 34 (77s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 19:57:02 (1736989022) fail_loc=0x80000317 Race attempt 0 Wait for 15425 15565 for 60 sec... total: 4000 open/close in 7.29 seconds: 548.34 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.201.109@tcp:/lustre 7.2G 14M 7.2G 1% /mnt/lustre 192.168.201.109@tcp:/lustre 7.2G 14M 7.2G 1% /mnt/lustre2 PASS 35 (63s) == sanityn test 36: handle ESTALE/open-unlink correctly == 19:58:05 (1736989085) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg109-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.281516 s, 186 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6833 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(0) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.294881 s, 178 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6833 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(1) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.275721 s, 190 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6833 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(2) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.278546 s, 188 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6833 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(3) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.28689 s, 183 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6833 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(4) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.280832 s, 187 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6833 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(5) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.313753 s, 167 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6833 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(6) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.278481 s, 188 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6833 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(7) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.380917 s, 138 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6833 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(8) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.251269 s, 209 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6833 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(9) *** before(7524352) after_dd(7473152) after(7524352) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.254199 s, 206 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.6833 sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete *** cycle(10) *** before(7524352) after_dd(7473152) after(7524352) PASS 36 (228s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 20:01:53 (1736989313) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.6833 total: 10000 create in 9.85 seconds: 1014.98 ops/second PASS 37 (13s) == sanityn test 39a: test from 11063 ============================================================================================ 20:02:06 (1736989326) repeat after cancel_lru_locks PASS 39a (2s) == sanityn test 39b: 11063 problem 1 ============================================================================================ 20:02:08 (1736989328) repeat after cancel_lru_locks PASS 39b (2s) == sanityn test 39c: check truncate mtime update ================================================================================ 20:02:11 (1736989331) repeat after cancel_lru_locks PASS 39c (3s) == sanityn test 39d: sync write should update mtime ====== 20:02:14 (1736989334) fail_loc=0x411 fail_loc=0 PASS 39d (2s) == sanityn test 40a: pdirops: create vs others ======================================================================== 20:02:16 (1736989336) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (8s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 20:02:24 (1736989344) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (7s) == sanityn test 40c: pdirops: link and others ======================================================================== 20:02:31 (1736989351) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (8s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 20:02:39 (1736989359) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (8s) == sanityn test 40e: pdirops: rename and others ======================================================================== 20:02:47 (1736989367) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (7s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 20:02:54 (1736989374) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (5s) == sanityn test 41b: pdirops: create vs create ======================================================================== 20:02:59 (1736989379) open: File exists Conflict PASS 41b (5s) == sanityn test 41c: pdirops: create vs link ======================================================================== 20:03:05 (1736989385) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (5s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 20:03:10 (1736989390) Conflict PASS 41d (5s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 20:03:15 (1736989395) Conflict PASS 41e (6s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 20:03:21 (1736989401) Conflict PASS 41f (5s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 20:03:26 (1736989406) Conflict PASS 41g (5s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 20:03:31 (1736989411) Conflict PASS 41h (5s) == sanityn test 41i: reint_open: create vs create ======== 20:03:36 (1736989416) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (663s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 20:14:39 (1736990079) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (4s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 20:14:43 (1736990083) open: File exists Conflict PASS 42b (5s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 20:14:48 (1736990088) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (4s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 20:14:52 (1736990092) Conflict PASS 42d (4s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 20:14:56 (1736990096) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (4s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 20:15:00 (1736990100) Conflict PASS 42f (4s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 20:15:04 (1736990104) Conflict PASS 42g (4s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 20:15:08 (1736990108) Conflict PASS 42h (4s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 20:15:12 (1736990112) PASS 43a (10s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 20:15:22 (1736990122) Conflict PASS 43b (4s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 20:15:26 (1736990126) Conflict PASS 43c (4s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 20:15:30 (1736990130) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (5s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 20:15:35 (1736990135) Conflict PASS 43e (4s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 20:15:39 (1736990139) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (4s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 20:15:43 (1736990143) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (4s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 20:15:47 (1736990147) Conflict PASS 43h (4s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 20:15:51 (1736990151) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (1s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 20:15:52 (1736990152) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists PASS 43j (33s) == sanityn test 43k: unlink vs create ==================== 20:16:25 (1736990185) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (382s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 20:22:47 (1736990567) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Wed Jan 15 20:22:50 EST 2025 PASS 44a (4s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 20:22:51 (1736990571) open: File exists Conflict PASS 44b (5s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 20:22:56 (1736990576) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (4s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 20:23:00 (1736990580) Conflict PASS 44d (4s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 20:23:04 (1736990584) Conflict PASS 44e (5s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 20:23:09 (1736990589) Conflict PASS 44f (4s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 20:23:13 (1736990593) Conflict PASS 44g (4s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 20:23:17 (1736990597) Conflict PASS 44h (4s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 20:23:21 (1736990601) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (1s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 20:23:22 (1736990602) PASS 45a (22s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 20:23:44 (1736990624) Conflict PASS 45b (4s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 20:23:48 (1736990628) Conflict PASS 45c (5s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 20:23:53 (1736990633) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (4s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 20:23:57 (1736990637) Conflict PASS 45e (5s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 20:24:02 (1736990642) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (4s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 20:24:06 (1736990646) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (5s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 20:24:11 (1736990651) Conflict PASS 45h (4s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 20:24:15 (1736990655) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (0s) == sanityn test 45j: read vs rename ====================== 20:24:16 (1736990656) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (356s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 20:30:12 (1736991012) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (4s) == sanityn test 46b: pdirops: link vs create ======================================================================== 20:30:16 (1736991016) open: File exists Conflict PASS 46b (4s) == sanityn test 46c: pdirops: link vs link ======================================================================== 20:30:20 (1736991020) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (5s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 20:30:25 (1736991025) Conflict PASS 46d (4s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 20:30:29 (1736991029) Conflict PASS 46e (4s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 20:30:33 (1736991033) Conflict PASS 46f (4s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 20:30:37 (1736991037) Conflict PASS 46g (5s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 20:30:42 (1736991042) Conflict PASS 46h (4s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 20:30:46 (1736991046) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (1s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 20:30:47 (1736991047) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (1s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 20:30:48 (1736991048) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (0s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 20:30:48 (1736991048) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (1s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 20:30:49 (1736991049) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (1s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 20:30:50 (1736991050) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (1s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 20:30:51 (1736991051) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (0s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 20:30:51 (1736991051) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (1s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 20:30:52 (1736991052) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.00407086 s, 2.5 MB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (7s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 20:30:59 (1736991059) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.00333292 s, 47.7 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3146: kill: (24371) - No such process PASS 51a (3s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 20:31:02 (1736991062) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0021825 s, 469 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126173 s, 83.1 MB/s 3145728 PASS 51b (14s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 20:31:16 (1736991076) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0036046 s, 284 kB/s PASS 51c (6s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 20:31:22 (1736991082) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972478 s, 108 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (2s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 20:31:24 (1736991084) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2437 0x985 0 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2437 0x985 0 read lease(1) has applied. PASS 51e (4s) == sanityn test 54: rename locking ======================= 20:31:28 (1736991088) llite.lustre-ffff8800ab31c000.xattr_cache=1 llite.lustre-ffff8800b3d07800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504963 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-15 20:31:28.000000000 -0500 Modify: 2025-01-15 20:31:28.000000000 -0500 Change: 2025-01-15 20:31:28.000000000 -0500 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504966 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-15 20:31:33.000000000 -0500 Modify: 2025-01-15 20:31:34.000000000 -0500 Change: 2025-01-15 20:31:34.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504969 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-15 20:31:39.000000000 -0500 Modify: 2025-01-15 20:31:39.000000000 -0500 Change: 2025-01-15 20:31:39.000000000 -0500 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504968 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-15 20:31:39.000000000 -0500 Modify: 2025-01-15 20:31:39.000000000 -0500 Change: 2025-01-15 20:31:39.000000000 -0500 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504971 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-15 20:31:44.000000000 -0500 Modify: 2025-01-15 20:31:44.000000000 -0500 Change: 2025-01-15 20:31:44.000000000 -0500 Birth: - File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272504972 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-15 20:31:44.000000000 -0500 Modify: 2025-01-15 20:31:44.000000000 -0500 Change: 2025-01-15 20:31:44.000000000 -0500 Birth: - PASS 54 (23s) == sanityn test 55a: rename vs unlink target dir ========= 20:31:51 (1736991111) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (7s) == sanityn test 55b: rename vs unlink source dir ========= 20:31:58 (1736991118) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (6s) == sanityn test 55c: rename vs unlink orphan target dir == 20:32:04 (1736991124) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (13s) == sanityn test 55d: rename file vs link ================= 20:32:17 (1736991137) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (8s) == sanityn test 60: Verify data_version behaviour ======== 20:32:25 (1736991145) PASS 60 (2s) == sanityn test 70a: cd directory && rm directory ======== 20:32:27 (1736991147) PASS 70a (1s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 20:32:28 (1736991148) PASS 70b (2s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 20:32:30 (1736991150) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00313172 s, 20.9 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (1s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 20:32:31 (1736991151) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00375229 s, 17.5 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (1s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 20:32:32 (1736991152) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (0s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 20:32:32 (1736991152) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00376427 s, 17.4 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (1s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 20:32:33 (1736991153) llite.lustre-ffff8800ab31c000.xattr_cache=1 llite.lustre-ffff8800b3d07800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (2s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 20:32:35 (1736991155) llite.lustre-ffff8800ab31c000.xattr_cache=1 llite.lustre-ffff8800b3d07800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (1s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 20:32:36 (1736991156) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00230028 s, 445 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00205674 s, 498 kB/s 3023: get lock2 3024: get lock1 3023: done 3023: get lock1 3024: done 3024: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 3024: cannot get lock2: Resource deadlock avoided 3024: exit rc=0 3023: done 3023: exit rc=0 PASS 74 (6s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 20:32:42 (1736991162) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0143404 s, 146 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00776342 s, 135 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3714: 3659 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (7s) == sanityn test 76: Verify MDT open_files listing ======== 20:32:49 (1736991169) opening files: ........ closing files: ........ PASS 76 (13s) == sanityn test 77a: check FIFO NRS policy =============== 20:33:02 (1736991182) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.094954 s, 177 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010699 s, 98.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125315 s, 83.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011646 s, 90.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210622 s, 49.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276074 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277479 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0296092 s, 35.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.032624 s, 32.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3d07800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0373902 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0485468 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0501377 s, 20.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0468284 s, 22.4 MB/s , 0.0543876 s, 19.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0428021 s, 24.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0484767 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0487236 s, 21.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115937 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110257 s, 95.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00267409 s, 392 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00926458 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120976 s, 86.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968599 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985135 s, 106 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116393 s, 90.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0135003 s, 77.7 MB/s , 0.0139869 s, 75.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01372 s, 76.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138292 s, 75.8 MB/s 1048576 bytes (1.0 MB) copied, 0.00368579 s, 284 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0148436 s, 70.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0133788 s, 78.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=6 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163562 s, 64.1 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg109-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (2s) == sanityn test 77b: check CRR-N NRS policy ============== 20:33:04 (1736991184) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.095219 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117497 s, 89.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125062 s, 83.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238386 s, 44.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0289435 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0381057 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0374232 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378445 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0438619 s, 23.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0380875 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0459774 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0380979 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0425628 s, 24.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.037441 s, 28.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.047437 s, 22.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3d07800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0395524 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0482482 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00758089 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00957685 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0088321 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109051 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00435469 s, 241 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00198486 s, 528 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00251666 s, 417 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00933169 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010768 s, 97.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00136075 s, 771 MB/s 1048576 bytes (1.0 MB) copied, 0.00885105 s, 118 MB/s , 0.0112489 s, 93.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00198855 s, 527 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00125591 s, 835 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00118606 s, 884 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142619 s, 73.5 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=2 removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg109-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0794365 s, 211 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00731638 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0087518 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150964 s, 69.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00855935 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0183988 s, 57.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0179137 s, 58.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120424 s, 87.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0277766 s, 37.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219799 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279872 s, 37.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0216139 s, 48.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0372408 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0310868 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229951 s, 45.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190101 s, 55.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0221483 s, 47.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00832864 s, 126 MB/s , 0.00887349 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129118 s, 81.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00957372 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110972 s, 94.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010372 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108067 s, 97.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112943 s, 92.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118417 s, 88.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117335 s, 89.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112537 s, 93.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120763 s, 86.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117195 s, 89.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0124531 s, 84.2 MB/s , 0.0116811 s, 89.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01207 s, 86.9 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg109-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (3s) == sanityn test 77c: check ORR NRS policy ================ 20:33:07 (1736991187) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.121147 s, 138 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00912821 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119427 s, 87.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271533 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0234661 s, 44.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0311521 s, 33.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331945 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259203 s, 40.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0196773 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261897 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211763 s, 49.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0369614 s, 28.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353547 s, 29.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0326273 s, 32.1 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3d07800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258567 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291436 s, 36.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192233 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0073008 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0079586 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0026309 s, 399 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102853 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00481133 s, 218 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00878924 s, 119 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00937961 s, 112 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00940077 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00895175 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114482 s, 91.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0121181 s, 86.5 MB/s , 0.0106829 s, 98.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107262 s, 97.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=1 1048576 bytes (1.0 MB) copied, 0.00801751 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123957 s, 84.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130716 s, 80.2 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg109-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0909358 s, 184 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111453 s, 94.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123586 s, 84.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142228 s, 73.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280448 s, 37.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0146981 s, 71.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188876 s, 55.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02981 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307236 s, 34.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0310251 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295376 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0409968 s, 25.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0276582 s, 37.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3d07800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392495 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.044093 s, 23.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0368963 s, 28.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0454769 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00166486 s, 630 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00901792 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00253323 s, 414 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00126533 s, 829 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0059715 s, 176 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00586509 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00640969 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00710882 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00719136 s, 146 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00818426 s, 128 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00914909 s, 115 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00848754 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0107909 s, 97.2 MB/s , 0.00242624 s, 432 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101547 s, 103 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105011 s, 99.9 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg109-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0884553 s, 190 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00659451 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011404 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138798 s, 75.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105887 s, 99.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015575 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00664174 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107551 s, 97.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182846 s, 57.3 MB/s 1048576 bytes (1.0 MB) copied, 0.017029 s, 61.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023517 s, 44.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282623 s, 37.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386077 s, 27.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0397644 s, 26.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0459482 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.045778 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0503519 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0084277 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00493742 s, 212 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00524318 s, 200 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00840947 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128667 s, 81.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00671499 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012169 s, 86.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0147846 s, 70.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116492 s, 90.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00903546 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00982255 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0110237 s, 95.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121507 s, 86.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013639 s, 76.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142097 s, 73.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139996 s, 74.9 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=3 removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg109-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (4s) == sanityn test 77d: check TRR nrs policy ================ 20:33:11 (1736991191) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.142325 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00864261 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125654 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017777 s, 59.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182332 s, 57.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292928 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232011 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208347 s, 50.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255684 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317635 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270297 s, 38.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0339302 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0260353 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219392 s, 47.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0262062 s, 40.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191413 s, 54.8 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3d07800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0287291 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00367907 s, 285 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00682126 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00388667 s, 270 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00810163 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00488102 s, 215 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00577276 s, 182 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127583 s, 82.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00501812 s, 209 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00752063 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00773053 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00786421 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00905014 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00951472 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0095821 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972797 s, 108 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011248 s, 93.2 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg109-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.139266 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00917481 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101012 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00935084 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129091 s, 81.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239694 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193931 s, 54.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229211 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332587 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0298639 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297137 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238423 s, 44.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3d07800.lock_unused_count=5 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0269297 s, 38.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333113 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0353384 s, 29.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464185 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346225 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00573806 s, 183 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00416263 s, 252 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00701697 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00548654 s, 191 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0053081 s, 198 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00829423 s, 126 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00849418 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00565301 s, 185 MB/s 1048576 bytes (1.0 MB) copied, 0.0096802 s, 108 MB/s 1048576 bytes (1.0 MB) copied, 0.00906294 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128305 s, 81.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00192764 s, 544 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00891572 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00853879 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0014238 s, 736 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00143964 s, 728 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg109-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0818377 s, 205 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00761649 s, 138 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0204784 s, 51.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0232042 s, 45.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0221286 s, 47.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0186844 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00881777 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181125 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0223531 s, 46.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129703 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0312534 s, 33.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0247368 s, 42.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242558 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0350038 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242652 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449082 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0358558 s, 29.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=7 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00745483 s, 141 MB/s , 0.00190187 s, 551 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00154516 s, 679 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00934461 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00838894 s, 125 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00938574 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114925 s, 91.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100395 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103469 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0102452 s, 102 MB/s 1048576 bytes (1.0 MB) copied, 0.00509314 s, 206 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0100211 s, 105 MB/s , 0.00729864 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011678 s, 89.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010527 s, 99.6 MB/s , 0.0116337 s, 90.1 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg109-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (3s) == sanityn test 77e: check TBF NID nrs policy ============ 20:33:14 (1736991194) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.201.9@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.109312 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00938069 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.014719 s, 71.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282449 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0290426 s, 36.1 MB/s , 0.0301994 s, 34.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318715 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236132 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365587 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0274385 s, 38.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0297683 s, 35.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149289 s, 70.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01694 s, 61.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0251069 s, 41.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0204494 s, 51.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253683 s, 41.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02445 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00715542 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0088102 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00684375 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00190154 s, 551 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0094925 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00954485 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00905264 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104603 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00974801 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104734 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00977078 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00947584 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101617 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105811 s, 99.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108093 s, 97.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115823 s, 90.5 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg109-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.107589 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00776209 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011818 s, 88.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0223355 s, 46.9 MB/s , 0.018012 s, 58.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201466 s, 52.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158705 s, 66.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263922 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306667 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282811 s, 37.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.033173 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0368255 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360043 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0325738 s, 32.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360676 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0409132 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0320324 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00740049 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00739772 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00945772 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00703351 s, 149 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100572 s, 104 MB/s 1048576 bytes (1.0 MB) copied, 0.00774783 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00950763 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00160155 s, 655 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00961821 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972867 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968978 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100089 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010557 s, 99.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00960725 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0106218 s, 98.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118416 s, 88.6 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg109-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0890866 s, 188 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0099015 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013464 s, 77.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0291808 s, 35.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0351006 s, 29.9 MB/s , 0.0229105 s, 45.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0374603 s, 28.0 MB/s , 0.0304054 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0313647 s, 33.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0289306 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0378844 s, 27.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268162 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0306273 s, 34.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.031294 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289238 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149804 s, 70.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=4 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3d07800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0385002 s, 27.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118771 s, 88.3 MB/s , 0.00852223 s, 123 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00139574 s, 751 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00707877 s, 148 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103123 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00985016 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00850894 s, 123 MB/s , 0.00813149 s, 129 MB/s , 0.010589 s, 99.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0106823 s, 98.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0099774 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00726053 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00161732 s, 648 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105909 s, 99.0 MB/s 1048576 bytes (1.0 MB) copied, 0.00192681 s, 544 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102835 s, 102 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg109-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.117892 s, 142 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00986727 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164351 s, 63.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0193478 s, 54.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0230338 s, 45.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0225628 s, 46.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0223599 s, 46.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0334488 s, 31.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117934 s, 88.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0176517 s, 59.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328142 s, 32.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219974 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263855 s, 39.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0369217 s, 28.4 MB/s , 0.0323794 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0126235 s, 83.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139694 s, 75.1 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00500759 s, 209 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00446796 s, 235 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00687119 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107363 s, 97.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00634356 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00750284 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0106101 s, 98.8 MB/s 1048576 bytes (1.0 MB) copied, 0.00766859 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00466701 s, 225 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00302029 s, 347 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00167885 s, 625 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00636946 s, 165 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100909 s, 104 MB/s 1048576 bytes (1.0 MB) copied, 0.00382218 s, 274 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00191108 s, 549 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00197649 s, 531 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg109-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (6s) == sanityn test 77f: check TBF JobID nrs policy ========== 20:33:20 (1736991200) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg109-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0929888 s, 180 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0090723 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00951881 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0143321 s, 73.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227556 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227148 s, 46.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0225534 s, 46.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0339143 s, 30.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331269 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0287606 s, 36.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231005 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255992 s, 41.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0298979 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0366144 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0310834 s, 33.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364523 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.039651 s, 26.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00994164 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113406 s, 92.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113823 s, 92.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0446598 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121715 s, 86.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0125194 s, 83.8 MB/s , 0.0122974 s, 85.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121082 s, 86.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out , 0.0127282 s, 82.4 MB/s 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0435526 s, 24.1 MB/s , 0.012774 s, 82.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0134675 s, 77.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0481773 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140108 s, 74.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0454428 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174232 s, 60.2 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg109-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg109-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0942572 s, 178 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00876418 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011271 s, 93.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0117648 s, 89.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235399 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259548 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261504 s, 40.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0185133 s, 56.6 MB/s , 0.0135282 s, 77.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0386573 s, 27.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0359772 s, 29.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177283 s, 59.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0305504 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0303742 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163373 s, 64.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294245 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00908856 s, 115 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00662715 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00941596 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0095658 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00878076 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104536 s, 100 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0115062 s, 91.1 MB/s , 0.0124433 s, 84.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00957055 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00999333 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010234 s, 102 MB/s , 0.01185 s, 88.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105485 s, 99.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0115013 s, 91.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116972 s, 89.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00982286 s, 107 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127723 s, 82.1 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg109-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg109-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0887766 s, 189 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00945972 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139316 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.021693 s, 48.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278236 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0327073 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0224495 s, 46.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0362423 s, 28.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0308087 s, 34.0 MB/s 1048576 bytes (1.0 MB) copied, 0.016422 s, 63.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0304004 s, 34.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.032999 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300267 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0299264 s, 35.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255432 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0163859 s, 64.0 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3d07800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263463 s, 39.8 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00144156 s, 727 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00138609 s, 756 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00651713 s, 161 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00139394 s, 752 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0196585 s, 53.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00142387 s, 736 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191765 s, 54.7 MB/s , 0.0183714 s, 57.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231214 s, 45.4 MB/s , 0.0153482 s, 68.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0168687 s, 62.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00241448 s, 434 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00809717 s, 129 MB/s , 0.00807403 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00587003 s, 179 MB/s 1048576 bytes (1.0 MB) copied, 0.00499764 s, 210 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg109-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg109-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.106315 s, 158 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0135099 s, 77.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0155895 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153906 s, 68.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139525 s, 75.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0348596 s, 30.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0289679 s, 36.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276316 s, 37.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0329757 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0417114 s, 25.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0437223 s, 24.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0403538 s, 26.0 MB/s 1+0 records in 1+0 records out , 0.0313814 s, 33.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0429679 s, 24.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0397869 s, 26.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=3 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3d07800.lock_unused_count=3 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0363832 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0437856 s, 23.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0065499 s, 160 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00732534 s, 143 MB/s 1048576 bytes (1.0 MB) copied, 0.0108976 s, 96.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00680122 s, 154 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00744573 s, 141 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00155099 s, 676 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00945766 s, 111 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00931365 s, 113 MB/s , 0.00922111 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0112524 s, 93.2 MB/s , 0.0115747 s, 90.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104405 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113668 s, 92.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128202 s, 81.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0131715 s, 79.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0133163 s, 78.7 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg109-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (6s) == sanityn test 77g: Change TBF type directly ============ 20:33:26 (1736991206) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (3s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 20:33:29 (1736991209) oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory pdsh@oleg109-client: oleg109-server: ssh exited with exit code 2 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 pdsh@oleg109-client: oleg109-server: ssh exited with exit code 12 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 pdsh@oleg109-client: oleg109-server: ssh exited with exit code 12 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 pdsh@oleg109-client: oleg109-server: ssh exited with exit code 12 PASS 77h (2s) == sanityn test 77i: Change rank of TBF rule ============= 20:33:31 (1736991211) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (6s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 20:33:37 (1736991217) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.0807133 s, 208 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00764265 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155506 s, 67.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180377 s, 58.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0185568 s, 56.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0149626 s, 70.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0182675 s, 57.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154879 s, 67.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011231 s, 93.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184452 s, 56.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109197 s, 96.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025896 s, 40.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176037 s, 59.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153312 s, 68.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0367795 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217452 s, 48.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025779 s, 40.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00658104 s, 159 MB/s , 0.00655955 s, 160 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00928442 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00533194 s, 197 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00684004 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0063827 s, 164 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0125787 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00749237 s, 140 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00833977 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0085525 s, 123 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0092552 s, 113 MB/s 1048576 bytes (1.0 MB) copied, 0.00940033 s, 112 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0103867 s, 101 MB/s , 0.0104678 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108707 s, 96.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011609 s, 90.3 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg109-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.93691 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4287 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (52s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 20:34:29 (1736991269) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg109-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.638287 s, 26.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.170832 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178545 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188832 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.174339 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.192915 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.177336 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.176855 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.192574 s, 5.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.373043 s, 2.8 MB/s , 0.371686 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.38002 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.378842 s, 2.8 MB/s , 0.384766 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.374311 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.391015 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.382279 s, 2.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.151332 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153312 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154243 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153938 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152934 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152609 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156241 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153737 s, 6.8 MB/s 1048576 bytes (1.0 MB) copied, 0.155172 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154742 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153977 s, 6.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.154431 s, 6.8 MB/s , 0.15538 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154448 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.154516 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157778 s, 6.6 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg109-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8268 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9746 s, 5.2 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg109-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.665042 s, 25.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.132177 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.126025 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.137918 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.114685 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.140322 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.127032 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.137817 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.129439 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.330314 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.326716 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.334182 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.330457 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.350177 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.353932 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.324623 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.353556 s, 3.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.13991 s, 7.5 MB/s , 0.141079 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.140323 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.140363 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.141239 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143749 s, 7.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142884 s, 7.3 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143179 s, 7.3 MB/s , 0.142677 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142261 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143474 s, 7.3 MB/s , 0.143334 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.144131 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.144164 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145128 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146778 s, 7.1 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg109-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8301 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9701 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (135s) == sanityn test 77k: check TBF policy with NID/JobID/OPCode expression ========================================================== 20:36:44 (1736991404) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg109-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.206107 s, 81.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162798 s, 64.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259712 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0332617 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238462 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364918 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012251 s, 85.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0178486 s, 58.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0218276 s, 48.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0529089 s, 19.8 MB/s , 0.044262 s, 23.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.055343 s, 18.9 MB/s , 0.0627318 s, 16.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0450594 s, 23.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0409619 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0633757 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0527372 s, 19.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00800764 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00887328 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118879 s, 88.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113916 s, 92.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00875399 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011596 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120744 s, 86.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0130567 s, 80.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011411 s, 91.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140012 s, 74.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01463 s, 71.7 MB/s , 0.0137777 s, 76.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161123 s, 65.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0160134 s, 65.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174165 s, 60.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0180295 s, 58.2 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg109-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.95218 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73991 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.9@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.9@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.18626 s, 90.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236964 s, 44.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215097 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139741 s, 75.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00899787 s, 117 MB/s 1048576 bytes (1.0 MB) copied, 0.0208272 s, 50.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0210242 s, 49.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172869 s, 60.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0177532 s, 59.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0734553 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0724422 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.160673 s, 6.5 MB/s , 0.166822 s, 6.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.272588 s, 3.8 MB/s , 0.260875 s, 4.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.366053 s, 2.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.350046 s, 3.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00756083 s, 139 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00579996 s, 181 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0061405 s, 171 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00732423 s, 143 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0103155 s, 102 MB/s , 0.00868496 s, 121 MB/s , 0.00782168 s, 134 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00895058 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00932386 s, 112 MB/s , 0.00832983 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0080397 s, 130 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0083034 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00925663 s, 113 MB/s 1048576 bytes (1.0 MB) copied, 0.0089779 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00878293 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00872224 s, 120 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg109-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92727 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.73133 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.9@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg109-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.193058 s, 86.9 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176529 s, 59.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0190872 s, 54.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0118318 s, 88.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288691 s, 36.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279853 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0081512 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164773 s, 63.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0209059 s, 50.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0636641 s, 16.5 MB/s , 0.0564465 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0728953 s, 14.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0498331 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0545984 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0665019 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.068062 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0747739 s, 14.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00704146 s, 149 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00685648 s, 153 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00858857 s, 122 MB/s 1048576 bytes (1.0 MB) copied, 0.00954133 s, 110 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00943193 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008729 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00864482 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00407277 s, 257 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0089098 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105286 s, 99.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00825392 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00920307 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00925443 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00916042 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102376 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108753 s, 96.4 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg109-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.91215 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.87543 s, 21.5 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg109-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.36065 s, 46.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139803 s, 75.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240081 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127037 s, 82.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128139 s, 81.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0296706 s, 35.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201595 s, 52.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0233589 s, 44.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239632 s, 43.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.142605 s, 7.4 MB/s , 0.14255 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14706 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152434 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.13918 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157047 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.341675 s, 3.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.335183 s, 3.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00969607 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0120597 s, 86.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111424 s, 94.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103913 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0116047 s, 90.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104977 s, 99.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0121936 s, 86.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0119952 s, 87.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123837 s, 84.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0114977 s, 91.2 MB/s , 0.011634 s, 90.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.012897 s, 81.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.013007 s, 80.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0139518 s, 75.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138036 s, 76.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140675 s, 74.5 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg109-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77529 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72918 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.90629 s, 21.4 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72859 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={1000} rate=5 running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg109-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.638358 s, 26.3 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00712367 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00670276 s, 156 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.01223 s, 85.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.00934594 s, 112 MB/s 1048576 bytes (1.0 MB) copied, 0.00824405 s, 127 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00756696 s, 139 MB/s , 0.00663035 s, 158 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0142892 s, 73.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.370465 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.384982 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.374317 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.370623 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.383614 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.388938 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.384864 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.382261 s, 2.7 MB/s running as uid/gid/euid/egid 500/1000/500/1000, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00581088 s, 180 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00771567 s, 136 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00785784 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.008291 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104817 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00882242 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00867614 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104969 s, 99.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00893374 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00973688 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00998987 s, 105 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104358 s, 100 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0103894 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010834 s, 96.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0108034 s, 97.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123334 s, 85.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg109-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8266 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/1000/500/1000, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.4288 s, 5.4 MB/s Read runtime is 20 s, speed is 5.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg109-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.187496 s, 89.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188372 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0310485 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0262766 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321434 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0433605 s, 24.2 MB/s , 0.0330796 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.031941 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0385147 s, 27.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0687075 s, 15.3 MB/s , 0.0667563 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0788672 s, 13.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0721497 s, 14.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0674093 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0737824 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0852026 s, 12.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0741696 s, 14.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00494831 s, 212 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00493086 s, 213 MB/s , 0.0050986 s, 206 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00856362 s, 122 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.010487 s, 100 MB/s , 0.00659384 s, 159 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00764074 s, 137 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00709304 s, 148 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109995 s, 95.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00787841 s, 133 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00887797 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00899205 s, 117 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00876492 s, 120 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00845692 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00858633 s, 122 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00970136 s, 108 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg109-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92514 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72978 s, 10.8 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg109-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.337815 s, 49.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_w] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00943912 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015168 s, 69.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0150424 s, 69.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109576 s, 95.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271301 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284068 s, 36.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189241 s, 55.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0189306 s, 55.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.166067 s, 6.3 MB/s , 0.168013 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.177951 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.177028 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.238614 s, 4.4 MB/s , 0.229508 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.228956 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.232781 s, 4.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: [declare] [-a] [pids_r] execvp fails running declare (2): No such file or directory 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00607977 s, 172 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0083004 s, 126 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00600413 s, 175 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00643014 s, 163 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00810545 s, 129 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00905777 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00715712 s, 147 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00803235 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00867034 s, 121 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00890864 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00893606 s, 117 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00803641 s, 130 MB/s , 0.00930991 s, 113 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00941441 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00972503 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.010342 s, 101 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg109-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.77854 s, 10.7 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.7298 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.78445 s, 10.7 MB/s Write runtime is 10 s, speed is 10.000000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.72917 s, 10.8 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (362s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 20:42:47 (1736991767) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (2s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 20:42:49 (1736991769) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg109-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.93853 s, 266 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg109-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0135 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (50s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 20:43:39 (1736991819) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.082968 s, 202 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0077669 s, 135 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0111375 s, 94.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0230046 s, 45.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017978 s, 58.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0158981 s, 66.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114636 s, 91.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294737 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00846586 s, 124 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0320257 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207292 s, 50.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258258 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282281 s, 37.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0243101 s, 43.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0207497 s, 50.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186234 s, 56.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0267658 s, 39.2 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b3d07800.lock_unused_count=1 ldlm.namespaces.lustre-OST0001-osc-ffff8800b3d07800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00921443 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00797432 s, 131 MB/s , 0.0017716 s, 592 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0113587 s, 92.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0105647 s, 99.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00971676 s, 108 MB/s , 0.00690191 s, 152 MB/s ldlm.namespaces.lustre-OST0001-osc-ffff8800b3d07800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00941894 s, 111 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00922776 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00992258 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00958997 s, 109 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00984772 s, 106 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102977 s, 102 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100517 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101031 s, 104 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0100921 s, 104 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg109-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.92029 s, 21.3 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96931 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.222718 s, 75.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172893 s, 60.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215474 s, 48.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0445523 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0436554 s, 24.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0648177 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0595875 s, 17.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0908872 s, 11.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0795195 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.134583 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145838 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169371 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.173127 s, 6.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.21941 s, 4.8 MB/s , 0.246033 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.282741 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.281454 s, 3.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154447 s, 67.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132463 s, 79.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186584 s, 56.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0154794 s, 67.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174157 s, 60.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0155832 s, 67.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0161017 s, 65.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164217 s, 63.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0172518 s, 60.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0207059 s, 50.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0188174 s, 55.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018792 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213384 s, 49.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208721 s, 50.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0222086 s, 47.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217953 s, 48.1 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg109-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg109-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.94191 s, 21.2 MB/s Write runtime is 6 s, speed is 16.666666 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.96766 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS zfs sleep 5 for ZFS zfs Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (67s) == sanityn test 77o: Changing rank should not panic ====== 20:44:46 (1736991886) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (3s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 20:44:49 (1736991889) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={500}&gid={500} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_policies=fifo PASS 77q (25s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 20:45:15 (1736991915) mds.MDS.mdt.nrs_policies=tbf oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long pdsh@oleg109-client: oleg109-server: ssh exited with exit code 36 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument pdsh@oleg109-client: oleg109-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (9s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 20:45:24 (1736991924) oleg109-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device pdsh@oleg109-client: oleg109-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (2s) == sanityn test 79: xattr: intent error ================== 20:45:26 (1736991926) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 131072 directory Device: 2c54f966h/743766374d Inode: 144115205272505431 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-15 20:45:27.000000000 -0500 Modify: 2025-01-15 20:45:27.000000000 -0500 Change: 2025-01-15 20:45:27.000000000 -0500 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (12s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 20:45:38 (1736991938) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (1s) == sanityn test 80b: Accessing directory during migration ========================================================== 20:45:39 (1736991939) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (0s) == sanityn test 81a: rename and stat under striped directory ========================================================== 20:45:39 (1736991939) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (1s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 20:45:40 (1736991940) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (1s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 20:45:41 (1736991941) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (1s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 20:45:42 (1736991942) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.6833 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.6833 PASS 82 (1s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 20:45:43 (1736991943) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (1s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 20:45:44 (1736991944) fail_loc=0x8000060b PASS 84 (7s) == sanityn test 90: open/create and unlink striped directory ========================================================== 20:45:51 (1736991951) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (0s) == sanityn test 91: chmod and unlink striped directory === 20:45:51 (1736991951) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (1s) == sanityn test 92: create remote directory under orphan directory ========================================================== 20:45:52 (1736991952) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (1s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 20:45:53 (1736991953) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2646 0xa56 0 1 2627 0xa43 0 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2628 0xa44 0 0 2647 0xa57 0 rc1=0 and rc2=0 PASS 93 (7s) == sanityn test 94: signal vs CP callback race =========== 20:46:01 (1736991961) osc.lustre-OST0000-osc-ffff8800ab31c000.idle_timeout=0 osc.lustre-OST0000-osc-ffff8800b3d07800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800ab31c000.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b3d07800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.0353749 s, 232 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 3.00168 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5135: 4998 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0126507 s, 324 kB/s osc.lustre-OST0000-osc-ffff8800ab31c000.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b3d07800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800ab31c000.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b3d07800.idle_timeout=debug PASS 94 (9s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 20:46:10 (1736991970) llite.lustre-ffff8800ab31c000.fast_read=0 llite.lustre-ffff8800b3d07800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00148382 s, 8.3 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800ab31c000.fast_read=1 llite.lustre-ffff8800b3d07800.fast_read=1 PASS 95b (8s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 20:46:18 (1736991978) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (0s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 20:46:18 (1736991978) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00137819 s, 3.0 MB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (2s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 20:46:20 (1736991980) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0204207 s, 103 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (1s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 20:46:21 (1736991981) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0206069 s, 102 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (2s) == sanityn test 100e: DoM: read on open and file size ==== 20:46:23 (1736991983) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00137474 s, 8.7 MB/s PASS 100e (1s) == sanityn test 101a: Discard DoM data on unlink ========= 20:46:24 (1736991984) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000505941 s, 8.1 MB/s PASS 101a (2s) == sanityn test 101b: Discard DoM data on rename ========= 20:46:26 (1736991986) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000463365 s, 8.8 MB/s PASS 101b (1s) == sanityn test 101c: Discard DoM data on close-unlink === 20:46:27 (1736991987) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000431672 s, 9.5 MB/s PASS 101c (3s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 20:46:30 (1736991990) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1736991991 lockahead_test: lockahead_test.c:1186: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1736991993 Finishing test test23 at 1736991993 PASS 103 (7s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 20:46:37 (1736991997) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (0s) == sanityn test 105: Glimpse and lock cancel race ======== 20:46:37 (1736991997) fail_loc=0x416 PASS 105 (47s) == sanityn test 106a: Verify the btime via statx() ======= 20:47:24 (1736992044) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (1s) == sanityn test 106b: Glimpse RPCs test for statx ======== 20:47:25 (1736992045) SKIP: sanityn test_106b statx() only test SKIP 106b (1s) == sanityn test 106c: Verify statx attributes mask ======= 20:47:26 (1736992046) SKIP: sanityn test_106c statx() only test SKIP 106c (0s) == sanityn test 107a: Basic grouplock conflict =========== 20:47:27 (1736992047) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0610603 s, 172 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 2633 0xa49 0 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6833 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.6833 First grouplock blocks second one PASS 107a (4s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 20:47:30 (1736992050) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.0549779 s, 191 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2653 0xa5d 0 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.6833 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (8s) == sanityn test 108a: lseek: parallel updates ============ 20:47:38 (1736992058) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.00144064 s, 6.9 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.22281 s, 2.0 MB/s PASS 108a (5s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 20:47:43 (1736992063) 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions oleg109-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg109-server: libkmod: kmod_module_get_size: failed to read coresize from /sys/module/intel_rapl Iteration 2 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 3 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 4 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions oleg109-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 5 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 6 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 7 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions oleg109-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 8 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 9 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions oleg109-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 10 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 11 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions oleg109-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 13 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 14 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 16 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 18 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 19 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 22 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 23 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 24 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 25 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 28 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 30 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 32 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 33 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions oleg109-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg109-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 34 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions oleg109-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 36 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 38 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 39 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 40 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 42 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 43 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 45 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 46 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 49 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Iteration 50 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre 192.168.201.109@tcp:/lustre /mnt/lustre lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 192.168.201.109@tcp:/lustre /mnt/lustre2 lustre rw,checksum,flock,user_xattr,lruresize,lazystatfs,nouser_fid2path,verbose,noencrypt 0 0 Stopping client oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre (opts:) LNET ready to unload Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg109-server' oleg109-server: oleg109-server.virtnet: executing load_modules_local oleg109-server: Loading modules from /home/green/git/lustre-release/lustre oleg109-server: detected 4 online CPUs by sysfs oleg109-server: Force libcfs to create 2 CPU partitions Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre Starting client: oleg109-client.virtnet: -o user_xattr,flock oleg109-server@tcp:/lustre /mnt/lustre2 PASS 109 (376s) == sanityn test 110: do not grant another lock on resend ========================================================== 20:53:59 (1736992439) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-01-15 20:53:59.000000000 -0500 Modify: 2025-01-15 20:53:59.000000000 -0500 Change: 2025-01-15 20:53:59.000000000 -0500 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg109-client.virtnet oleg109-client.virtnet PASS 110 (89s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 20:55:28 (1736992528) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (1s) == sanityn test 112: update max-inherit in default LMV === 20:55:29 (1736992529) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (1s) == sanityn test 113: check servers of specified fs ======= 20:55:30 (1736992530) lustre-OST0000-osc-ffff8800ab102800 active. lustre-OST0001-osc-ffff8800ab102800 active. lustre-OST0000-osc-ffff8800ab102800 active. lustre-OST0001-osc-ffff8800ab102800 active. MGC192.168.201.109@tcp active. PASS 113 (2s) cleanup: ====================================================== == sanityn test complete, duration 4029 sec ============== 20:55:32 (1736992532) Stopping clients: oleg109-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg109-client.virtnet /mnt/lustre2 opts: