-----============= acceptance-small: sanityn ============----- Thu May 1 20:43:39 EDT 2025 mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_ID=7 CLIENT_OS_VERSION_CODE=117440512 oleg120-server: ls: cannot access /home/green/git/lustre-release/lustre/tests/except/sanityn.*ex: No such file or directory excepting tests: 27 28 102 skipping tests SLOW=no: 33a === sanityn: start setup 20:43:51 (1746146631) === Starting client oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Started clients oleg120-client.virtnet: 192.168.201.120@tcp:/lustre on /mnt/lustre2 type lustre (rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose) oleg120-client.virtnet: executing check_config_client /mnt/lustre oleg120-client.virtnet: Checking config lustre mounted on /mnt/lustre Checking servers environments Checking clients oleg120-client.virtnet environments Using TIMEOUT=20 osc.lustre-OST0000-osc-ffff8800aa215800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b6e70800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aa215800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b6e70800.idle_timeout=debug disable quota as required oleg120-server: oleg120-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all === sanityn: finish setup 20:44:04 (1746146644) === 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171567 s, 61.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [true] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d0_runas_test/f7050] == sanityn test 0: do_node_vp() and do_facet_vp() do the right thing ========================================================== 20:44:06 (1746146646) PASS 0 (4s) == sanityn test 1: Check attribute updates on 2 mount points ========================================================== 20:44:10 (1746146650) /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0777 OK /mnt/lustre/f1.sanityn has type file OK /mnt/lustre/f1.sanityn has perms 0666 OK /mnt/lustre/f1.sanityn: absent OK PASS 1 (3s) == sanityn test 2a: check cached attribute updates on 2 mtpt's ================================================================== 20:44:13 (1746146653) -rw-r--r-- 1 root root 0 May 1 20:44 /mnt/lustre2/f2a /mnt/lustre/f2a has type file OK /mnt/lustre/f2a has perms 0777 OK PASS 2a (3s) == sanityn test 2b: check cached attribute updates on 2 mtpt's ================================================================== 20:44:16 (1746146656) -rw-r--r-- 1 root root 0 May 1 20:44 /mnt/lustre2/f2b /mnt/lustre2/f2b has type file OK /mnt/lustre2/f2b has perms 0777 OK PASS 2b (3s) == sanityn test 2c: check cached attribute updates on 2 mtpt's root ============================================================= 20:44:19 (1746146659) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0777 OK PASS 2c (3s) == sanityn test 2d: check cached attribute updates on 2 mtpt's root ============================================================= 20:44:22 (1746146662) /mnt/lustre2 has type dir OK /mnt/lustre2 has perms 0755 OK PASS 2d (3s) == sanityn test 2e: check chmod on root is propagated to others ========================================================== 20:44:26 (1746146666) total 1 -rwxrwxrwx 1 root root 0 May 1 20:44 f2a -rwxrwxrwx 1 root root 0 May 1 20:44 f2b total 1 -rwxrwxrwx 1 root root 0 May 1 20:44 f2a -rwxrwxrwx 1 root root 0 May 1 20:44 f2b running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre2/f2e.sanityn] [count=1] 1+0 records in 1+0 records out 512 bytes (512 B) copied, 0.0068777 s, 74.4 kB/s PASS 2e (4s) == sanityn test 2f: check attr/owner updates on DNE with 2 mtpt's ========================================================== 20:44:29 (1746146669) SKIP: sanityn test_2f needs >= 2 MDTs SKIP 2f (2s) == sanityn test 2g: check blocks update on sync write ==== 20:44:31 (1746146671) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.381357 s, 5.5 MB/s /mnt/lustre/f2g.sanityn has 4103 blocks /mnt/lustre2/f2g.sanityn has 4103 blocks PASS 2g (4s) == sanityn test 3: symlink on one mtpt, readlink on another ===================================================================== 20:44:35 (1746146675) PASS 3 (3s) == sanityn test 4: fstat validation on multiple mount points ==================================================================== 20:44:38 (1746146678) PASS 4 (5s) == sanityn test 5: create a file on one mount, truncate it on the other ========================================================== 20:44:43 (1746146683) /mnt/lustre/f5 has type file OK /mnt/lustre/f5 has size 100 OK PASS 5 (4s) == sanityn test 6: remove of open file on other node ============================================================================ 20:44:47 (1746146687) opening writing unlinking /mnt/lustre2/f6.sanityn accessing (1) seeking (1) accessing (2) fstat... reading comparing data truncating seeking (2) writing again seeking (3) reading again comparing data again closing SUCCESS - goto beer PASS 6 (4s) == sanityn test 7: remove of open directory on other node ======================================================================= 20:44:51 (1746146691) creating directory /mnt/lustre/d7 opening directory unlinking /mnt/lustre/d7 Ok, everything goes well. PASS 7 (4s) == sanityn test 8: remove of open special file on other node ==================================================================== 20:44:55 (1746146695) creating special file /mnt/lustre/f8.sanityn opening file unlinking /mnt/lustre/f8.sanityn Ok, everything goes well. PASS 8 (4s) == sanityn test 9a: append of file with sub-page size on multiple mounts ========================================================== 20:44:59 (1746146699) PASS 9a (5s) == sanityn test 9b: append to striped sparse file ======== 20:45:04 (1746146704) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249523 s, 42.0 MB/s 3+0 records in 3+0 records out 3 bytes (3 B) copied, 0.0408714 s, 0.1 kB/s Data read (expecting 'foo'): foo PASS 9b (4s) == sanityn test 10a: write of file with sub-page size on multiple mounts ========================================================== 20:45:08 (1746146708) 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00823478 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00936746 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00680494 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00525022 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00833205 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0110976 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00646903 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00720099 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00499913 s, 0.2 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.00386647 s, 0.3 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0149638 s, 0.1 kB/s 1+0 records in 1+0 records out 1 byte (1 B) copied, 0.0123221 s, 0.1 kB/s PASS 10a (5s) == sanityn test 10b: write of file with sub-page size on multiple mounts ========================================================== 20:45:13 (1746146713) 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.0136599 s, 225 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0289225 s, 142 kB/s 1+0 records in 1+0 records out 3072 bytes (3.1 kB) copied, 0.000285557 s, 10.8 MB/s PASS 10b (3s) == sanityn test 11: execution of file opened for write should return error ============================================================== 20:45:17 (1746146717) multiop /mnt/lustre/d11/f vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 301: /mnt/lustre2/d11/f: Text file busy PASS 11 (4s) == sanityn test 12: test lock ordering (link, stat, unlink) ========================================================== 20:45:20 (1746146720) warning: '-runtime' deprecated, use '-t runtime' instead - link 572 (time 1746146731.93 total 10.00 last 57.19) - link 1144 (time 1746146741.93 total 20.01 last 57.17) - link 1703 (time 1746146751.94 total 30.01 last 55.89) - link 2288 (time 1746146761.95 total 40.02 last 58.43) - link 2812 (time 1746146771.96 total 50.04 last 52.32) - link 3553 (time 1746146781.97 total 60.04 last 74.04) - link 4234 (time 1746146791.97 total 70.05 last 68.09) - link 4911 (time 1746146801.98 total 80.05 last 67.64) - link 5521 (time 1746146812.01 total 90.08 last 60.85) total: 6100 link in 100.01 seconds: 61.00 ops/second using seed 4072740984 running for 100 seconds - stat 10000 (time 1746146732 ; total 9 ; last 9) - stat 20000 (time 1746146742 ; total 19 ; last 10) - stat 30000 (time 1746146751 ; total 28 ; last 9) - stat 40000 (time 1746146760 ; total 37 ; last 9) - stat 50000 (time 1746146770 ; total 47 ; last 10) - stat 55099 (time 1746146781 ; total 58 ; last 11) - stat 57734 (time 1746146792 ; total 69 ; last 11) - stat 60000 (time 1746146801 ; total 78 ; last 9) - stat 62183 (time 1746146812 ; total 89 ; last 11) - stat 64972 (time 1746146823 ; total 100 ; last 11) total: 64973 stats in 100 seconds: 649.729980 stats/second - unlinked 0 (time 1746146773 ; total 1 ; last 1) unlink(/mnt/lustre2/lockdir/lockfile6100) error: No such file or directory total: 6099 unlinks in 93 seconds: 65.580643 unlinks/second /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 77: kill: (22105) - No such process /home/green/git/lustre-release/lustre/tests/lockorder.sh: line 78: kill: (22108) - No such process PASS 12 (148s) == sanityn test 13: test directory page revocation ======= 20:47:49 (1746146869) f13 f13 PASS 13 (5s) == sanityn test 14aa: execution of file open for write returns -ETXTBSY ========================================================== 20:47:54 (1746146874) multiop /mnt/lustre/d14aa.sanityn/f14aa.sanityn vOw_c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 336: /mnt/lustre2/d14aa.sanityn/f14aa.sanityn: Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4880: 23509 Terminated $MULTIOP_PROG $FILE v$ARGS > $TMPPIPE (wd: ~) PASS 14aa (4s) == sanityn test 14ab: open(RDWR) of executing file returns -ETXTBSY ========================================================== 20:47:58 (1746146878) open(O_RDWR|O_CREAT): Text file busy /home/green/git/lustre-release/lustre/tests/test-framework.sh: line 4880: 24146 Terminated $DIR1/$tdir/sleep 60 (wd: ~) PASS 14ab (5s) == sanityn test 14b: truncate of executing file returns -ETXTBSY ================================================================ 20:48:03 (1746146883) truncate: cannot truncate '/mnt/lustre2/d14b.sanityn/sleep' to length 60: Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 351: 24781 Terminated $DIR1/$tdir/sleep 60 PASS 14b (4s) == sanityn test 14c: open(O_TRUNC) of executing file return -ETXTBSY ============================================================ 20:48:07 (1746146887) cp: cannot create regular file '/mnt/lustre2/d14c.sanityn/sleep': Text file busy /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 363: 25418 Terminated $DIR1/$tdir/sleep 60 PASS 14c (4s) == sanityn test 14d: chmod of executing file is still possible ================================================================== 20:48:11 (1746146891) chmod /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 374: 26054 Terminated $DIR1/$tdir/sleep 60 PASS 14d (5s) == sanityn test 15: test out-of-space with multiple writers ===================================================================== 20:48:16 (1746146896) PATH=/opt/iozone/bin:/opt/iozone/bin:/home/green/git/lustre-release/lustre/tests/mpi:/home/green/git/lustre-release/lustre/tests/racer:/home/green/git/lustre-release/lustre/../lustre-iokit/sgpdd-survey:/home/green/git/lustre-release/lustre/tests:/home/green/git/lustre-release/lustre/utils/gss:/home/green/git/lustre-release/lustre/utils:/opt/iozone/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin: mgs: CentOS Linux release 7.9.2009 (Core) MGS_OS_VERSION_ID=7 MGS_OS_ID=centos MGS_OS_VERSION_CODE=117440512 MGS_OS_ID_LIKE=rhel fedora centos mds1: CentOS Linux release 7.9.2009 (Core) MDS1_OS_ID_LIKE=rhel fedora centos MDS1_OS_ID=centos MDS1_OS_VERSION_ID=7 MDS1_OS_VERSION_CODE=117440512 ost1: CentOS Linux release 7.9.2009 (Core) OST1_OS_VERSION_CODE=117440512 OST1_OS_VERSION_ID=7 OST1_OS_ID_LIKE=rhel fedora centos OST1_OS_ID=centos client: CentOS Linux release 7.9.2009 (Core) CLIENT_OS_ID=centos CLIENT_OS_ID_LIKE=rhel fedora centos CLIENT_OS_VERSION_CODE=117440512 CLIENT_OS_VERSION_ID=7 STRIPECOUNT=2 ORIGFREE=7521280 MAXFREE=800000 SKIP: oos2 test_15 oos2.sh: 7521280kB free gt MAXFREE 800000kB, increase 800000 (or reduce test fs size) to proceed sleep 5 for ZFS MDS Waiting for MDT destroys to complete SKIP 15 (22s) == sanityn test 16a: 500 iterations of dual-mount fsx ==== 20:48:38 (1746146918) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.440748 s, 19.0 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.295608 s, 28.4 MB/s Chance of close/open is 1 in 50 Seed set to 6920 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read 000004[1] 1746146921.622200 mapwrite 0x3875bb thru 0x39110a (0x9b50 bytes) 000007[0] 1746146921.911560 write 0x5b0542 thru 0x5b6148 (0x5c07 bytes) 000008[1] 1746146921.950269 write 0x045402 thru 0x04cf99 (0x7b98 bytes) 000010[1] 1746146921.988330 mapwrite 0x77838b thru 0x784de9 (0xca5f bytes) 000012[1] 1746146922.191244 read 0x5651f1 thru 0x568b7c (0x398c bytes) 000013[1] 1746146922.198150 read 0x4e4909 thru 0x4e7fba (0x36b2 bytes) 000014[0] 1746146922.224297 mapread 0x1d53a9 thru 0x1e4461 (0xf0b9 bytes) 000015[0] 1746146922.313333 write 0x6c0734 thru 0x6c4eaf (0x477c bytes) 000016[0] 1746146922.382241 write 0x42a87a thru 0x42e2eb (0x3a72 bytes) 000017[0] 1746146922.394043 read 0x3bf1e2 thru 0x3bf2a9 (0xc8 bytes) 000018[1] 1746146922.401890 mapwrite 0x6b3377 thru 0x6b75b4 (0x423e bytes) 000020[1] 1746146922.558731 mapwrite 0x7df916 thru 0x7eca36 (0xd121 bytes) 000021[1] 1746146922.749416 mapread 0x12d015 thru 0x12d1f2 (0x1de bytes) 000022[0] 1746146922.782304 read 0x174fb4 thru 0x17ac10 (0x5c5d bytes) truncating to largest ever: 0x34ebb9 000024[0] 1746146922.810523 trunc from 0x7eca37 to 0x34ebb8 (0x49de7e bytes) 000027[1] 1746146922.880190 read 0x2b93a3 thru 0x2c5a91 (0xc6ef bytes) 000028[0] 1746146922.983772 mapwrite 0x0a3c92 thru 0x0a6f82 (0x32f1 bytes) 000032[0] 1746146923.149224 write 0x8e91ce thru 0x8f2aec (0x991f bytes) 000038[0] 1746146923.176468 read 0x5ee91d thru 0x5f0a41 (0x2125 bytes) 000042[0] 1746146923.228224 write 0x917a2e thru 0x922c90 (0xb263 bytes) truncating to largest ever: 0x426689 000043[1] 1746146923.250317 trunc from 0x922c91 to 0x426688 (0x4fc608 bytes) 000044[0] 1746146923.316985 write 0x7566b8 thru 0x760645 (0x9f8e bytes) 000045[0] 1746146923.345183 trunc from 0x760646 to 0x1a51d1 (0x5bb474 bytes) 000047[1] 1746146923.371309 read 0x18c754 thru 0x19b482 (0xed2f bytes) 000048[1] 1746146923.398559 read 0x09b4e7 thru 0x0aa736 (0xf250 bytes) truncating to largest ever: 0x5936c0 000050[0] 1746146923.441495 trunc from 0x1a51d2 to 0x5936bf (0x3ee4ee bytes) 000054[1] 1746146923.559565 trunc from 0x5936c0 to 0x00a47d (0x589242 bytes) 000055[1] 1746146923.637832 read 0x0070fb thru 0x00a47d (0x3383 bytes) 000062[1] 1746146923.672941 mapwrite 0x8e1736 thru 0x8e80a6 (0x6971 bytes) 000063[1] 1746146923.918454 write 0x064c89 thru 0x06f328 (0xa6a0 bytes) 000066[0] 1746146923.940715 write 0x6fd2a2 thru 0x6fee2b (0x1b8a bytes) 000068[0] 1746146923.996379 mapread 0x15cfdc thru 0x169e40 (0xce65 bytes) 000070[0] 1746146924.094248 mapread 0x64fa10 thru 0x6547fa (0x4deb bytes) 000071[1] 1746146924.123214 mapread 0x78eeb7 thru 0x799cda (0xae24 bytes) 000074[1] 1746146924.217463 write 0x634fb5 thru 0x642c6d (0xdcb9 bytes) 000075[1] 1746146924.245333 read 0x84a9d2 thru 0x84e257 (0x3886 bytes) 000076[1] 1746146924.265940 read 0x8b5fd0 thru 0x8ba6e3 (0x4714 bytes) truncating to largest ever: 0x74535c 000077[1] 1746146924.277770 trunc from 0x8e80a7 to 0x74535b (0x1a2d4b bytes) truncating to largest ever: 0x8e60f3 000078[0] 1746146924.318323 trunc from 0x74535c to 0x8e60f2 (0x1a0d97 bytes) 000080[0] 1746146924.385495 write 0x6156e7 thru 0x622184 (0xca9e bytes) 000081[0] 1746146924.415085 read 0x15582a thru 0x1595e4 (0x3dbb bytes) 000085[0] 1746146924.454973 mapwrite 0x74c6d9 thru 0x75b2d0 (0xebf8 bytes) 000087[0] 1746146924.544425 mapread 0x571f71 thru 0x578858 (0x68e8 bytes) 000088[0] 1746146924.561387 mapwrite 0x40f1d5 thru 0x41607b (0x6ea7 bytes) 000089[1] 1746146924.650526 mapwrite 0x22fd5e thru 0x235dcc (0x606f bytes) 000090[0] 1746146924.823866 write 0x89a754 thru 0x89c79a (0x2047 bytes) 000092[1] 1746146924.854302 read 0x7b4af9 thru 0x7c310d (0xe615 bytes) 000093[1] 1746146924.887017 trunc from 0x8e60f3 to 0x64b939 (0x29a7b9 bytes) 000094[0] 1746146924.937980 mapread 0x34a977 thru 0x34b9c2 (0x104c bytes) 000098[0] 1746146924.993031 write 0x0e22f1 thru 0x0ee70b (0xc41b bytes) 000099[1] 1746146925.026945 mapwrite 0x394dc3 thru 0x3a1d46 (0xcf84 bytes) 000101[0] 1746146925.157710 write 0x89f35f thru 0x8a957d (0xa21f bytes) 000103[0] 1746146925.211160 mapwrite 0x396560 thru 0x39ca78 (0x6519 bytes) 000104[0] 1746146925.335044 read 0x3a9a2d thru 0x3add06 (0x42da bytes) 000106[0] 1746146925.381009 write 0x68ac36 thru 0x695c98 (0xb063 bytes) 000108[1] 1746146925.402595 read 0x24e66e thru 0x25a7a2 (0xc135 bytes) truncating to largest ever: 0x9cc2ba 000110[0] 1746146925.447232 trunc from 0x8a957e to 0x9cc2b9 (0x122d3c bytes) 000111[1] 1746146925.499961 trunc from 0x9cc2ba to 0x16a62f (0x861c8a bytes) 000115[1] 1746146925.584392 trunc from 0x16a630 to 0x118273 (0x523bc bytes) 000117[0] 1746146925.633596 write 0x52f745 thru 0x52f82d (0xe9 bytes) 000118[1] 1746146925.679677 read 0x19b819 thru 0x1a2839 (0x7021 bytes) 000119[1] 1746146925.775434 mapwrite 0x66d11e thru 0x670822 (0x3705 bytes) 000120[0] 1746146925.882353 mapread 0x59aa6c thru 0x5a384e (0x8de3 bytes) 000121[0] 1746146925.930502 mapwrite 0x84c03b thru 0x84f8a6 (0x386c bytes) 000124[1] 1746146926.164381 mapread 0x7f4b84 thru 0x7f4e3b (0x2b8 bytes) 000127[1] 1746146926.288864 trunc from 0x84f8a7 to 0x5c112b (0x28e77b bytes) 000128[1] 1746146926.350981 read 0x44cef7 thru 0x44e613 (0x171d bytes) 000129[1] 1746146926.358537 write 0x23e247 thru 0x243822 (0x55dc bytes) 000134[1] 1746146926.465483 read 0x02282f thru 0x0310ae (0xe880 bytes) 000135[1] 1746146926.491333 mapread 0x10c175 thru 0x117991 (0xb81d bytes) 000136[1] 1746146926.533386 read 0x50ff61 thru 0x511631 (0x16d1 bytes) 000139[0] 1746146926.556891 mapread 0x59c569 thru 0x5ab1bd (0xec55 bytes) 000140[1] 1746146926.599263 mapread 0x53a877 thru 0x53cd1c (0x24a6 bytes) 000141[1] 1746146926.614419 trunc from 0x5c112c to 0x18ae5f (0x4362cc bytes) 000143[0] 1746146926.684872 mapwrite 0x53b1f0 thru 0x53f045 (0x3e56 bytes) 000144[1] 1746146926.828447 write 0x596bec thru 0x59f074 (0x8489 bytes) 000145[0] 1746146926.855609 write 0x333b06 thru 0x33eea1 (0xb39c bytes) 000146[1] 1746146926.907512 write 0x00c1ca thru 0x00caec (0x923 bytes) 000149[0] 1746146926.934251 trunc from 0x59f075 to 0x1778cc (0x4277a8 bytes) 000151[1] 1746146927.014930 mapread 0x068aea thru 0x06f90b (0x6e22 bytes) 000153[1] 1746146927.052376 read 0x019b41 thru 0x01bed5 (0x2395 bytes) 000155[1] 1746146927.112019 mapread 0x0bc651 thru 0x0c3779 (0x7129 bytes) 000156[0] 1746146927.133230 trunc from 0x1778cd to 0x154c37 (0x22c95 bytes) 000157[0] 1746146927.176146 write 0x53c26e thru 0x54b05b (0xedee bytes) 000158[0] 1746146927.178260 mapwrite 0x668e2a thru 0x66dc40 (0x4e17 bytes) 000164[0] 1746146927.341609 write 0x991182 thru 0x9944b5 (0x3334 bytes) 000165[0] 1746146927.368406 mapread 0x6fbdef thru 0x6fd556 (0x1768 bytes) 000166[1] 1746146927.408402 mapread 0x0a9275 thru 0x0b360f (0xa39b bytes) 000168[1] 1746146927.572447 read 0x0f03db thru 0x0fdd0a (0xd930 bytes) 000169[1] 1746146927.594485 mapread 0x398e88 thru 0x3a666d (0xd7e6 bytes) 000170[0] 1746146927.615192 mapread 0x25e9e0 thru 0x265ef1 (0x7512 bytes) 000171[0] 1746146927.664189 write 0x502bc7 thru 0x511f34 (0xf36e bytes) 000173[1] 1746146927.737555 write 0x65cd60 thru 0x66060b (0x38ac bytes) 000176[1] 1746146927.796055 write 0x230c79 thru 0x2391bb (0x8543 bytes) 000177[0] 1746146927.843682 mapread 0x8e1c8d thru 0x8f1123 (0xf497 bytes) 000178[0] 1746146927.891982 read 0x1597bf thru 0x15bc67 (0x24a9 bytes) 000186[0] 1746146927.935392 read 0x1e35e2 thru 0x1ee042 (0xaa61 bytes) 000188[1] 1746146927.957331 mapread 0x93b248 thru 0x944003 (0x8dbc bytes) 000189[1] 1746146927.975220 mapwrite 0x131951 thru 0x13aea3 (0x9553 bytes) 000191[1] 1746146928.112300 read 0x946b42 thru 0x950507 (0x99c6 bytes) 000192[0] 1746146928.118161 read 0x2e2db5 thru 0x2f166e (0xe8ba bytes) 000194[1] 1746146928.142653 trunc from 0x9944b6 to 0x1ce911 (0x7c5ba4 bytes) 000195[1] 1746146928.201773 write 0x70ccae thru 0x70f56e (0x28c1 bytes) 000196[0] 1746146928.234085 trunc from 0x70f56f to 0x283298 (0x48c2d6 bytes) 000199[0] 1746146928.288721 write 0x630cd8 thru 0x632db6 (0x20df bytes) 000200[0] 1746146928.291265 trunc from 0x632db7 to 0x22867e (0x40a738 bytes) 000203[1] 1746146928.325089 mapread 0x1704d2 thru 0x1737d8 (0x3307 bytes) 000205[1] 1746146928.353434 read 0x18abe8 thru 0x1959a2 (0xadbb bytes) 000206[0] 1746146928.399185 write 0x1ab121 thru 0x1b3d45 (0x8c25 bytes) 000208[0] 1746146928.476459 write 0x2ccf59 thru 0x2d67d3 (0x987b bytes) 000210[1] 1746146928.480396 mapwrite 0x18293a thru 0x18ccbb (0xa382 bytes) 000214[1] 1746146928.688937 mapread 0x2600f0 thru 0x260570 (0x481 bytes) 000218[1] 1746146928.777497 read 0x2cda68 thru 0x2d67d3 (0x8d6c bytes) 000220[0] 1746146928.800120 mapwrite 0x2a3f6a thru 0x2a5b1b (0x1bb2 bytes) 000221[1] 1746146928.867960 write 0x2214c6 thru 0x230725 (0xf260 bytes) 000222[1] 1746146928.924450 read 0x127c1f thru 0x135760 (0xdb42 bytes) 000224[0] 1746146928.963634 read 0x1fc1c9 thru 0x20a89d (0xe6d5 bytes) 000227[1] 1746146929.052859 mapwrite 0x6b7666 thru 0x6c1c35 (0xa5d0 bytes) 000228[0] 1746146929.229751 write 0x916dda thru 0x922943 (0xbb6a bytes) 000230[0] 1746146929.282490 mapwrite 0x39ae89 thru 0x39e9a0 (0x3b18 bytes) 000231[1] 1746146929.387915 trunc from 0x922944 to 0x081c67 (0x8a0cdc bytes) 000236[0] 1746146929.436796 mapwrite 0x24aabe thru 0x254019 (0x955c bytes) 000238[1] 1746146929.620314 mapread 0x2072ed thru 0x20fbfd (0x8911 bytes) 000239[1] 1746146929.697985 read 0x00525c thru 0x0131c0 (0xdf65 bytes) 000241[0] 1746146929.751044 read 0x0163f4 thru 0x01ffb1 (0x9bbe bytes) 000243[1] 1746146929.804971 mapread 0x1dd5ec thru 0x1e5dbe (0x87d3 bytes) 000244[0] 1746146929.813047 trunc from 0x25401a to 0x122f96 (0x131083 bytes) 000246[1] 1746146929.896218 mapread 0x0e8f34 thru 0x0f69fa (0xdac7 bytes) 000248[1] 1746146929.941057 read 0x0985ba thru 0x0a7d4e (0xf795 bytes) 000250[0] 1746146929.978208 read 0x11a6d5 thru 0x122f96 (0x88c2 bytes) 000251[0] 1746146930.000709 write 0x2bd416 thru 0x2c2929 (0x5514 bytes) 000254[0] 1746146930.040185 mapwrite 0x313785 thru 0x31fde7 (0xc663 bytes) 000257[1] 1746146930.173127 read 0x1e288f thru 0x1ec885 (0x9ff7 bytes) 000259[0] 1746146930.287229 write 0x8c9812 thru 0x8cd48e (0x3c7d bytes) 000260[1] 1746146930.307929 mapwrite 0x71f910 thru 0x72abf4 (0xb2e5 bytes) 000262[1] 1746146930.651635 mapread 0x5fdcd0 thru 0x6086bc (0xa9ed bytes) 000263[0] 1746146930.700159 mapwrite 0x05d2f4 thru 0x06ac29 (0xd936 bytes) 000264[0] 1746146930.847823 trunc from 0x8cd48f to 0x5f7499 (0x2d5ff5 bytes) 000265[0] 1746146930.877517 mapread 0x2cd416 thru 0x2dc5bf (0xf1aa bytes) 000267[0] 1746146930.965877 trunc from 0x5f749a to 0x6405bc (0x49123 bytes) 000268[1] 1746146930.981472 trunc from 0x6405bd to 0x1e95a4 (0x457018 bytes) 000269[0] 1746146931.066250 read 0x1a4662 thru 0x1b2081 (0xda20 bytes) 000270[0] 1746146931.139703 write 0x803124 thru 0x805834 (0x2711 bytes) 000272[0] 1746146931.193006 trunc from 0x805835 to 0x263feb (0x5a1849 bytes) 000273[0] 1746146931.224166 write 0x96f9e0 thru 0x97d6f8 (0xdd19 bytes) 000276[1] 1746146931.282503 write 0x82b3dd thru 0x82e267 (0x2e8b bytes) 000279[0] 1746146931.379281 trunc from 0x97d6f9 to 0x4843ca (0x4f932e bytes) 000280[1] 1746146931.428868 write 0x46f853 thru 0x47eeb8 (0xf666 bytes) 000285[1] 1746146931.518175 mapread 0x33d829 thru 0x33e27a (0xa52 bytes) 000286[1] 1746146931.550497 mapread 0x40d573 thru 0x41abe5 (0xd673 bytes) 000287[0] 1746146931.551288 mapread 0x03a495 thru 0x03ce42 (0x29ae bytes) 000288[1] 1746146931.611142 mapread 0x1f3afe thru 0x1f8c0e (0x5111 bytes) 000289[0] 1746146931.695540 read 0x074e60 thru 0x07d400 (0x85a1 bytes) 000290[0] 1746146931.712634 write 0x18f692 thru 0x197478 (0x7de7 bytes) 000291[1] 1746146931.764641 write 0x68c50b thru 0x6964e9 (0x9fdf bytes) 000293[1] 1746146931.805349 read 0x2f105d thru 0x2f387d (0x2821 bytes) 000295[0] 1746146931.845531 mapwrite 0x171b2d thru 0x179aa9 (0x7f7d bytes) 000300[0] 1746146932.030897 mapwrite 0x388614 thru 0x390ba5 (0x8592 bytes) 000304[0] 1746146932.154463 mapwrite 0x058fb8 thru 0x059376 (0x3bf bytes) 000307[1] 1746146932.253520 write 0x1f6ecb thru 0x1f7515 (0x64b bytes) 000308[1] 1746146932.293452 trunc from 0x6964ea to 0x3b9133 (0x2dd3b6 bytes) 000310[0] 1746146932.339141 mapwrite 0x2af9e7 thru 0x2b0db8 (0x13d2 bytes) 000311[1] 1746146932.459480 mapwrite 0x29d9cd thru 0x2a213e (0x4772 bytes) 000315[1] 1746146932.595976 read 0x1a6412 thru 0x1b434b (0xdf3a bytes) 000316[1] 1746146932.658473 mapread 0x32f4dc thru 0x3357a0 (0x62c5 bytes) 000318[0] 1746146932.684218 write 0x0a6064 thru 0x0a9396 (0x3333 bytes) 000319[1] 1746146932.742232 read 0x2d151b thru 0x2dc689 (0xb16f bytes) 000323[0] 1746146932.879803 mapwrite 0x51b9ae thru 0x51ecad (0x3300 bytes) 000324[0] 1746146933.053998 trunc from 0x51ecae to 0x46cf0b (0xb1da2 bytes) 000326[0] 1746146933.092799 mapwrite 0x895d0a thru 0x89f61f (0x9916 bytes) 000327[0] 1746146933.228959 write 0x494963 thru 0x4a0e27 (0xc4c5 bytes) 000330[0] 1746146933.253828 mapread 0x78ac7c thru 0x78e681 (0x3a06 bytes) 000338[0] 1746146933.329625 mapread 0x5a8657 thru 0x5b5d0e (0xd6b8 bytes) 000339[0] 1746146933.353475 mapwrite 0x9a19bd thru 0x9af440 (0xda84 bytes) 000343[1] 1746146933.567186 trunc from 0x9af441 to 0x2e075b (0x6cece5 bytes) 000344[0] 1746146933.654856 read 0x1a9e40 thru 0x1b8f0b (0xf0cc bytes) 000345[1] 1746146933.733132 write 0x88bc1e thru 0x8969ab (0xad8e bytes) 000346[0] 1746146933.762678 trunc from 0x8969ac to 0x38d574 (0x509437 bytes) 000348[0] 1746146933.821155 mapread 0x1bf2a3 thru 0x1c5fe2 (0x6d40 bytes) 000354[0] 1746146933.923620 read 0x00fd9a thru 0x019827 (0x9a8e bytes) 000359[0] 1746146933.998291 read 0x162390 thru 0x16d8c7 (0xb538 bytes) 000360[1] 1746146934.015091 mapwrite 0x48f08f thru 0x49b758 (0xc6ca bytes) 000361[0] 1746146934.151110 mapread 0x088132 thru 0x090775 (0x8644 bytes) 000362[1] 1746146934.279448 read 0x087719 thru 0x08d545 (0x5e2d bytes) 000365[0] 1746146934.304149 trunc from 0x49b759 to 0x95188c (0x4b6134 bytes) 000366[1] 1746146934.395531 read 0x5dca55 thru 0x5e482d (0x7dd9 bytes) 000368[0] 1746146934.437885 write 0x00207a thru 0x011971 (0xf8f8 bytes) 000369[1] 1746146934.463957 read 0x4c8d01 thru 0x4ce21e (0x551e bytes) 000370[1] 1746146934.485013 read 0x429313 thru 0x42c97b (0x3669 bytes) 000371[0] 1746146934.489490 trunc from 0x95188d to 0x640bd7 (0x310cb5 bytes) 000373[1] 1746146934.516186 read 0x3380e2 thru 0x33a1f4 (0x2113 bytes) 000375[0] 1746146934.553166 trunc from 0x640bd8 to 0x7ae917 (0x16dd40 bytes) 000377[0] 1746146934.616647 trunc from 0x7ae918 to 0x0fc135 (0x6b27e2 bytes) 000379[0] 1746146934.635999 mapread 0x069d13 thru 0x079548 (0xf836 bytes) 000382[1] 1746146934.807134 trunc from 0x0fc136 to 0x032a73 (0xc96c2 bytes) 000384[0] 1746146934.879764 mapread 0x02a56e thru 0x032a73 (0x8506 bytes) 000385[1] 1746146934.956325 mapread 0x00276e thru 0x003c02 (0x1495 bytes) 000391[1] 1746146935.018816 write 0x0af414 thru 0x0ba5ef (0xb1dc bytes) 000392[1] 1746146935.031417 mapwrite 0x5884ac thru 0x58de98 (0x59ed bytes) 000394[0] 1746146935.142790 mapread 0x57b604 thru 0x585c78 (0xa675 bytes) 000396[1] 1746146935.250222 mapread 0x407dfe thru 0x413ac4 (0xbcc7 bytes) 000397[1] 1746146935.295633 trunc from 0x58de99 to 0x35263a (0x23b85e bytes) 000398[0] 1746146935.346652 read 0x21472c thru 0x2232dc (0xebb1 bytes) 000401[0] 1746146935.479749 mapread 0x0b992c thru 0x0c679c (0xce71 bytes) 000402[1] 1746146935.502785 write 0x3e16d6 thru 0x3effa8 (0xe8d3 bytes) 000403[1] 1746146935.548836 write 0x61c239 thru 0x62453e (0x8306 bytes) 000407[0] 1746146935.577317 write 0x68628f thru 0x694806 (0xe578 bytes) 000408[0] 1746146935.608230 trunc from 0x694807 to 0x7e030f (0x14bb09 bytes) 000409[0] 1746146935.668272 mapwrite 0x9415a3 thru 0x94d965 (0xc3c3 bytes) 000410[1] 1746146935.857758 write 0x2feea7 thru 0x305539 (0x6693 bytes) 000411[1] 1746146935.892834 mapread 0x867b6e thru 0x875e3e (0xe2d1 bytes) 000414[1] 1746146935.991713 write 0x41eebb thru 0x4214b4 (0x25fa bytes) 000415[1] 1746146936.020825 mapwrite 0x78e4f6 thru 0x79391d (0x5428 bytes) 000416[1] 1746146936.112776 read 0x4e990e thru 0x4f7d41 (0xe434 bytes) 000417[1] 1746146936.128540 read 0x77140c thru 0x7761f3 (0x4de8 bytes) 000420[1] 1746146936.163990 mapread 0x2a6539 thru 0x2a7729 (0x11f1 bytes) 000422[1] 1746146936.172496 write 0x255f82 thru 0x25cc1c (0x6c9b bytes) 000423[1] 1746146936.205993 mapread 0x5d889c thru 0x5e672f (0xde94 bytes) 000424[0] 1746146936.229232 read 0x4e0456 thru 0x4e871c (0x82c7 bytes) 000425[0] 1746146936.286774 write 0x439218 thru 0x445239 (0xc022 bytes) 000426[1] 1746146936.335557 trunc from 0x94d966 to 0x0c6b62 (0x886e03 bytes) 000427[1] 1746146936.390443 read 0x0a2ecb thru 0x0a7c03 (0x4d39 bytes) 000430[0] 1746146936.476297 read 0x0aa883 thru 0x0b0345 (0x5ac3 bytes) 000431[1] 1746146936.511848 write 0x9d8b61 thru 0x9ddad2 (0x4f72 bytes) 000432[1] 1746146936.558150 mapwrite 0x18f970 thru 0x197b84 (0x8215 bytes) 000433[0] 1746146936.668465 read 0x6ac1af thru 0x6ac6d5 (0x527 bytes) 000434[1] 1746146936.707436 read 0x1c42cb thru 0x1cc241 (0x7f77 bytes) 000435[0] 1746146936.707854 write 0x5f145a thru 0x5f8a8e (0x7635 bytes) 000436[0] 1746146936.754738 trunc from 0x9ddad3 to 0x74ba08 (0x2920ca bytes) 000438[1] 1746146936.808222 mapwrite 0x2b77a6 thru 0x2b9777 (0x1fd2 bytes) 000443[0] 1746146936.952379 trunc from 0x74ba09 to 0x6be7ff (0x8d209 bytes) 000444[1] 1746146937.019899 read 0x6865df thru 0x6933a7 (0xcdc9 bytes) 000447[1] 1746146937.095829 trunc from 0x6be800 to 0x2b6b92 (0x407c6d bytes) 000448[1] 1746146937.127436 read 0x131a80 thru 0x139926 (0x7ea7 bytes) 000449[0] 1746146937.212057 read 0x085baa thru 0x092ba1 (0xcff8 bytes) 000450[0] 1746146937.287520 mapread 0x0ad734 thru 0x0bb7a3 (0xe070 bytes) 000452[1] 1746146937.298329 read 0x23a339 thru 0x24724a (0xcf12 bytes) 000454[0] 1746146937.375162 trunc from 0x2b6b93 to 0x6a67e6 (0x3efc54 bytes) 000455[0] 1746146937.417786 mapwrite 0x7f2e05 thru 0x7fd468 (0xa664 bytes) 000457[0] 1746146937.582170 mapwrite 0x8cffb8 thru 0x8d5866 (0x58af bytes) 000459[1] 1746146937.705821 read 0x31c41d thru 0x325691 (0x9275 bytes) 000460[1] 1746146937.734609 mapwrite 0x0397d2 thru 0x04521f (0xba4e bytes) 000462[1] 1746146937.856841 trunc from 0x8d5867 to 0x15b907 (0x779f5f bytes) 000464[0] 1746146937.888860 mapread 0x02738d thru 0x02d04a (0x5cbe bytes) 000467[0] 1746146937.960796 mapread 0x0653d0 thru 0x073bf5 (0xe826 bytes) 000468[1] 1746146938.068923 write 0x8a5b7d thru 0x8b1c23 (0xc0a7 bytes) 000471[1] 1746146938.108251 write 0x48bc47 thru 0x49169b (0x5a55 bytes) 000472[0] 1746146938.147931 trunc from 0x8b1c24 to 0x6582f0 (0x259933 bytes) 000473[0] 1746146938.207836 mapwrite 0x4ab5e6 thru 0x4b89d6 (0xd3f1 bytes) 000474[1] 1746146938.339567 mapread 0x49aa7f thru 0x4a65a8 (0xbb2a bytes) 000478[1] 1746146938.376307 mapread 0x3cfac6 thru 0x3d231f (0x285a bytes) 000480[0] 1746146938.405879 write 0x13258e thru 0x13355f (0xfd2 bytes) 000482[0] 1746146938.429339 read 0x27aaa5 thru 0x27efdd (0x4539 bytes) 000483[0] 1746146938.451801 trunc from 0x6582f1 to 0x08d3a9 (0x5caf47 bytes) 000484[1] 1746146938.478940 mapwrite 0x2a37cd thru 0x2b0a4e (0xd282 bytes) 000485[1] 1746146938.611840 write 0x2fdfb0 thru 0x2ff711 (0x1762 bytes) 000486[0] 1746146938.637256 trunc from 0x2ff712 to 0x024e21 (0x2da8f0 bytes) truncating to largest ever: 0x9ee892 000487[1] 1746146938.695002 trunc from 0x024e22 to 0x9ee891 (0x9c9a70 bytes) 000488[0] 1746146938.743076 mapwrite 0x3c0cf5 thru 0x3c2df0 (0x20fc bytes) 000489[0] 1746146938.860428 read 0x605f8d thru 0x60efc5 (0x9039 bytes) 000491[1] 1746146938.898856 write 0x6d3bf8 thru 0x6e252a (0xe933 bytes) 000492[0] 1746146938.917315 write 0x8a1263 thru 0x8ae4f8 (0xd296 bytes) 000493[1] 1746146938.934003 mapread 0x5696e6 thru 0x56e848 (0x5163 bytes) 000494[0] 1746146938.978503 trunc from 0x9ee892 to 0x453da5 (0x59aaec bytes) 000496[1] 1746146939.023681 read 0x2d1558 thru 0x2d8868 (0x7311 bytes) 000498[1] 1746146939.055195 trunc from 0x453da6 to 0x6649ed (0x210c48 bytes) 000500[0] 1746146939.116609 mapwrite 0x63d07e thru 0x642a0b (0x598e bytes) All operations completed A-OK! Chance of close/open is 1 in 50 Seed set to 6939 fd 0: /mnt/lustre/f16a.sanityn fd 1: /mnt/lustre2/f16a.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000002[0] 1746146939.864028 write_OD 0x46f000 thru 0x472fff (0x4000 bytes) truncating to largest ever: 0x1f5885 000003[1] 1746146939.947144 trunc from 0x473000 to 0x1f5884 (0x27d77b bytes) 000004[1] 1746146939.978656 write_OD 0x3f4000 thru 0x3f6fff (0x3000 bytes) 000005[0] 1746146940.042915 write_OD 0x059000 thru 0x05ffff (0x7000 bytes) 000008[1] 1746146940.108876 write_OD 0x5eb000 thru 0x5edfff (0x3000 bytes) 000009[0] 1746146940.149755 mapread 0x230000 thru 0x23aec8 (0xaec9 bytes) 000011[0] 1746146940.195181 trunc from 0x5ee000 to 0x0c9634 (0x5249cb bytes) truncating to largest ever: 0x227d28 000012[0] 1746146940.213661 trunc from 0x0c9635 to 0x227d27 (0x15e6f3 bytes) 000013[0] 1746146940.226623 trunc from 0x227d28 to 0x102188 (0x125b9f bytes) 000015[1] 1746146940.254161 mapwrite 0x494000 thru 0x49656d (0x256e bytes) 000017[1] 1746146940.415796 mapread 0x3fb000 thru 0x402eb9 (0x7eba bytes) 000019[1] 1746146940.423294 mapwrite 0x5b2000 thru 0x5b5dc8 (0x3dc9 bytes) 000022[1] 1746146940.514871 write_OD 0x33b000 thru 0x33dfff (0x3000 bytes) 000024[1] 1746146940.552522 read_OD 0x30d000 thru 0x319fff (0xd000 bytes) 000025[1] 1746146940.558793 read_OD 0x45d000 thru 0x469fff (0xd000 bytes) truncating to largest ever: 0x36a5b9 000026[0] 1746146940.566443 trunc from 0x5b5dc9 to 0x36a5b8 (0x24b810 bytes) 000027[1] 1746146940.595582 read_OD 0x299000 thru 0x2a4fff (0xc000 bytes) 000031[0] 1746146940.619009 write_OD 0x66e000 thru 0x679fff (0xc000 bytes) 000032[0] 1746146940.695203 mapwrite 0x746000 thru 0x750fb1 (0xafb2 bytes) 000033[0] 1746146940.782399 mapwrite 0x7f0000 thru 0x7f57c2 (0x57c3 bytes) 000034[1] 1746146940.868134 read_OD 0x092000 thru 0x096fff (0x5000 bytes) 000035[0] 1746146940.881485 mapread 0x468000 thru 0x46a3a7 (0x23a8 bytes) truncating to largest ever: 0x49e32f 000036[1] 1746146940.904882 trunc from 0x7f57c3 to 0x49e32e (0x357494 bytes) 000037[0] 1746146940.921479 mapwrite 0x09a000 thru 0x09a44a (0x44b bytes) truncating to largest ever: 0x8e6f21 000038[1] 1746146940.984889 trunc from 0x49e32f to 0x8e6f20 (0x448bf2 bytes) 000039[1] 1746146941.005602 write_OD 0x463000 thru 0x46ffff (0xd000 bytes) 000041[0] 1746146941.051801 mapwrite 0x716000 thru 0x71865a (0x265b bytes) 000042[1] 1746146941.121683 trunc from 0x8e6f21 to 0x18c6b7 (0x75a869 bytes) 000045[0] 1746146941.156591 write_OD 0x1fa000 thru 0x203fff (0xa000 bytes) 000046[0] 1746146941.200342 write_OD 0x65e000 thru 0x65efff (0x1000 bytes) 000047[0] 1746146941.247338 write_OD 0x0a6000 thru 0x0abfff (0x6000 bytes) 000048[0] 1746146941.289646 mapwrite 0x891000 thru 0x892518 (0x1519 bytes) 000049[0] 1746146941.340650 mapread 0x150000 thru 0x154595 (0x4596 bytes) truncating to largest ever: 0x9f4e02 000052[1] 1746146941.381835 trunc from 0x892519 to 0x9f4e01 (0x1628e9 bytes) 000053[1] 1746146941.404921 mapread 0x73e000 thru 0x7403c0 (0x23c1 bytes) 000054[0] 1746146941.429034 trunc from 0x9f4e02 to 0x4c5c8e (0x52f173 bytes) 000055[1] 1746146941.451870 write_OD 0x15f000 thru 0x164fff (0x6000 bytes) 000056[1] 1746146941.496444 mapread 0x393000 thru 0x39e4ae (0xb4af bytes) 000057[0] 1746146941.508626 mapread 0x0b7000 thru 0x0ba909 (0x390a bytes) 000058[0] 1746146941.517233 trunc from 0x4c5c8f to 0x4a9f9e (0x1bcf0 bytes) 000059[0] 1746146941.541733 mapwrite 0x677000 thru 0x68492f (0xd930 bytes) 000062[1] 1746146941.628827 trunc from 0x684930 to 0x0824c1 (0x60246e bytes) 000065[0] 1746146941.655620 mapread 0x06f000 thru 0x073c3b (0x4c3c bytes) 000068[1] 1746146941.684125 mapread 0x07e000 thru 0x08154e (0x354f bytes) 000070[1] 1746146941.694700 mapread 0x047000 thru 0x052767 (0xb768 bytes) 000073[0] 1746146941.707719 mapwrite 0x0ff000 thru 0x1000dd (0x10de bytes) 000075[0] 1746146941.804777 mapwrite 0x314000 thru 0x320093 (0xc094 bytes) 000076[1] 1746146941.879436 mapread 0x051000 thru 0x057dc9 (0x6dca bytes) 000078[1] 1746146941.906752 write_OD 0x736000 thru 0x737fff (0x2000 bytes) 000079[1] 1746146941.945848 mapwrite 0x976000 thru 0x97df8a (0x7f8b bytes) 000081[0] 1746146942.066517 trunc from 0x97df8b to 0x7e5517 (0x198a73 bytes) 000082[0] 1746146942.108823 mapread 0x013000 thru 0x0168d3 (0x38d4 bytes) 000083[0] 1746146942.134932 mapread 0x004000 thru 0x007937 (0x3938 bytes) 000089[0] 1746146942.186618 trunc from 0x7e5518 to 0x9a109f (0x1bbb88 bytes) 000090[1] 1746146942.206201 read_OD 0x8fa000 thru 0x904fff (0xb000 bytes) 000091[1] 1746146942.228263 mapread 0x0cc000 thru 0x0d3c12 (0x7c13 bytes) 000096[1] 1746146942.254682 write_OD 0x574000 thru 0x57efff (0xb000 bytes) 000097[0] 1746146942.334566 mapwrite 0x9c1000 thru 0x9c21e9 (0x11ea bytes) 000098[1] 1746146942.421740 write_OD 0x3aa000 thru 0x3affff (0x6000 bytes) 000099[1] 1746146942.466412 trunc from 0x9c21ea to 0x648924 (0x3798c5 bytes) 000106[1] 1746146942.556307 mapread 0x125000 thru 0x1348e7 (0xf8e8 bytes) 000109[1] 1746146942.582932 read_OD 0x149000 thru 0x157fff (0xf000 bytes) 000110[0] 1746146942.604936 mapwrite 0x762000 thru 0x76cb6e (0xab6f bytes) 000111[1] 1746146942.743268 trunc from 0x76cb6f to 0x93c166 (0x1cf5f8 bytes) 000112[1] 1746146942.770224 read_OD 0x73e000 thru 0x748fff (0xb000 bytes) 000113[0] 1746146942.781713 write_OD 0x0ef000 thru 0x0f2fff (0x4000 bytes) 000115[0] 1746146942.837627 write_OD 0x8c7000 thru 0x8cbfff (0x5000 bytes) 000116[1] 1746146942.879427 trunc from 0x93c167 to 0x561ea8 (0x3da2be bytes) 000117[0] 1746146942.898608 read_OD 0x3b4000 thru 0x3b7fff (0x4000 bytes) 000118[1] 1746146942.922843 trunc from 0x561ea9 to 0x5b0765 (0x4e8bd bytes) 000119[1] 1746146942.953295 read_OD 0x3ce000 thru 0x3dafff (0xd000 bytes) 000121[0] 1746146942.970989 write_OD 0x37c000 thru 0x384fff (0x9000 bytes) 000122[1] 1746146943.031006 write_OD 0x09b000 thru 0x0a3fff (0x9000 bytes) 000123[0] 1746146943.075755 read_OD 0x1e8000 thru 0x1eefff (0x7000 bytes) 000124[1] 1746146943.078924 mapwrite 0x409000 thru 0x4104a1 (0x74a2 bytes) 000125[1] 1746146943.168391 trunc from 0x5b0766 to 0x1159a4 (0x49adc1 bytes) 000126[1] 1746146943.193904 mapread 0x06a000 thru 0x0771a7 (0xd1a8 bytes) 000134[1] 1746146943.248597 mapwrite 0x263000 thru 0x2689a4 (0x59a5 bytes) 000137[1] 1746146943.365908 trunc from 0x2689a5 to 0x57adec (0x312448 bytes) 000139[0] 1746146943.381228 trunc from 0x57aded to 0x5969be (0x1bbd2 bytes) 000140[1] 1746146943.407645 mapwrite 0x5ab000 thru 0x5b8229 (0xd22a bytes) 000142[0] 1746146943.514008 trunc from 0x5b822a to 0x886b02 (0x2ce8d9 bytes) 000145[1] 1746146943.537970 trunc from 0x886b03 to 0x3a8923 (0x4de1df bytes) 000146[0] 1746146943.565069 trunc from 0x3a8924 to 0x8206a5 (0x477d82 bytes) 000147[1] 1746146943.596651 read_OD 0x014000 thru 0x014fff (0x1000 bytes) 000153[1] 1746146943.621325 trunc from 0x8206a6 to 0x64176f (0x1def36 bytes) 000154[0] 1746146943.645726 write_OD 0x55f000 thru 0x565fff (0x7000 bytes) 000156[1] 1746146943.690341 mapread 0x2f1000 thru 0x2f8c44 (0x7c45 bytes) 000159[1] 1746146943.703505 read_OD 0x375000 thru 0x37bfff (0x7000 bytes) 000160[0] 1746146943.708536 mapread 0x544000 thru 0x55024b (0xc24c bytes) 000161[0] 1746146943.730684 mapread 0x503000 thru 0x50e052 (0xb053 bytes) 000162[1] 1746146943.741697 mapread 0x30e000 thru 0x311b60 (0x3b61 bytes) 000166[1] 1746146943.751488 read_OD 0x327000 thru 0x331fff (0xb000 bytes) 000169[1] 1746146943.759718 mapwrite 0x6aa000 thru 0x6b96d8 (0xf6d9 bytes) 000170[0] 1746146943.851045 write_OD 0x8f3000 thru 0x8fbfff (0x9000 bytes) 000172[0] 1746146943.906082 trunc from 0x8fc000 to 0x5153e0 (0x3e6c1f bytes) 000177[0] 1746146943.987633 mapwrite 0x50b000 thru 0x514cfb (0x9cfc bytes) 000180[1] 1746146944.076893 read_OD 0x407000 thru 0x409fff (0x3000 bytes) 000182[1] 1746146944.094699 trunc from 0x5153e1 to 0x54af7a (0x35b9a bytes) 000183[0] 1746146944.120838 mapread 0x18b000 thru 0x191037 (0x6038 bytes) 000184[1] 1746146944.159330 read_OD 0x523000 thru 0x52bfff (0x9000 bytes) 000185[1] 1746146944.165892 mapread 0x1d1000 thru 0x1dea94 (0xda95 bytes) 000186[0] 1746146944.186716 mapwrite 0x2c3000 thru 0x2c3521 (0x522 bytes) 000190[0] 1746146944.263137 read_OD 0x0f2000 thru 0x0f8fff (0x7000 bytes) 000191[1] 1746146944.278578 trunc from 0x54af7b to 0x8fb95c (0x3b09e2 bytes) 000192[0] 1746146944.296565 mapwrite 0x3c2000 thru 0x3cd6c5 (0xb6c6 bytes) 000195[1] 1746146944.414564 mapread 0x28c000 thru 0x29ae6c (0xee6d bytes) 000197[1] 1746146944.440603 mapwrite 0x2ef000 thru 0x2fb5b0 (0xc5b1 bytes) 000198[1] 1746146944.522497 read_OD 0x279000 thru 0x283fff (0xb000 bytes) 000201[0] 1746146944.540080 mapread 0x8bd000 thru 0x8cb71d (0xe71e bytes) 000202[0] 1746146944.557933 trunc from 0x8fb95d to 0x6377dd (0x2c417f bytes) 000205[1] 1746146944.599843 read_OD 0x021000 thru 0x022fff (0x2000 bytes) 000210[1] 1746146944.630289 write_OD 0x36f000 thru 0x372fff (0x4000 bytes) 000212[1] 1746146944.760850 write_OD 0x665000 thru 0x66bfff (0x7000 bytes) 000214[1] 1746146944.829192 write_OD 0x9b1000 thru 0x9b1fff (0x1000 bytes) 000216[0] 1746146944.904201 mapwrite 0x26a000 thru 0x2791f5 (0xf1f6 bytes) 000217[1] 1746146945.034559 write_OD 0x811000 thru 0x811fff (0x1000 bytes) 000219[1] 1746146945.088248 mapread 0x010000 thru 0x013d78 (0x3d79 bytes) 000221[1] 1746146945.100557 mapwrite 0x3cf000 thru 0x3cfdc3 (0xdc4 bytes) 000224[1] 1746146945.182016 trunc from 0x9b2000 to 0x8f1d95 (0xc026a bytes) 000226[1] 1746146945.211382 trunc from 0x8f1d96 to 0x8801dc (0x71bb9 bytes) 000233[1] 1746146945.296068 mapread 0x5fb000 thru 0x609f03 (0xef04 bytes) 000237[0] 1746146945.360763 read_OD 0x5da000 thru 0x5e4fff (0xb000 bytes) 000239[1] 1746146945.385591 trunc from 0x8801dd to 0x41a1fa (0x465fe2 bytes) 000240[0] 1746146945.418615 read_OD 0x2bc000 thru 0x2cafff (0xf000 bytes) 000241[0] 1746146945.444081 mapread 0x30f000 thru 0x31cb4c (0xdb4d bytes) 000246[0] 1746146945.466380 read_OD 0x1ff000 thru 0x206fff (0x8000 bytes) 000247[0] 1746146945.475512 mapwrite 0x8e9000 thru 0x8f2351 (0x9352 bytes) 000248[0] 1746146945.594890 read_OD 0x2ab000 thru 0x2b4fff (0xa000 bytes) 000253[1] 1746146945.621567 read_OD 0x3b2000 thru 0x3bffff (0xe000 bytes) 000256[0] 1746146945.629380 write_OD 0x80c000 thru 0x811fff (0x6000 bytes) 000257[0] 1746146945.680626 mapread 0x3ff000 thru 0x406573 (0x7574 bytes) 000261[0] 1746146945.691203 write_OD 0x45f000 thru 0x462fff (0x4000 bytes) 000262[0] 1746146945.719484 write_OD 0x0f1000 thru 0x0fdfff (0xd000 bytes) 000263[1] 1746146945.767831 trunc from 0x8f2352 to 0x527e44 (0x3ca50d bytes) 000264[0] 1746146945.792160 mapwrite 0x715000 thru 0x71df07 (0x8f08 bytes) 000266[0] 1746146945.903114 read_OD 0x436000 thru 0x43bfff (0x6000 bytes) 000267[0] 1746146945.925766 read_OD 0x368000 thru 0x374fff (0xd000 bytes) 000268[0] 1746146945.933892 read_OD 0x5af000 thru 0x5bcfff (0xe000 bytes) 000273[0] 1746146945.946970 mapread 0x386000 thru 0x38d67c (0x767d bytes) 000276[0] 1746146945.965419 read_OD 0x048000 thru 0x04cfff (0x5000 bytes) 000278[1] 1746146945.971035 mapwrite 0x44b000 thru 0x454815 (0x9816 bytes) 000287[1] 1746146946.139701 mapread 0x364000 thru 0x368ce1 (0x4ce2 bytes) 000288[1] 1746146946.151414 mapread 0x1cf000 thru 0x1d233b (0x333c bytes) 000292[1] 1746146946.203899 mapread 0x310000 thru 0x310b7b (0xb7c bytes) 000297[1] 1746146946.226740 mapread 0x673000 thru 0x67f0bb (0xc0bc bytes) 000300[1] 1746146946.261348 mapread 0x360000 thru 0x36bfe0 (0xbfe1 bytes) 000301[1] 1746146946.267711 mapread 0x1f8000 thru 0x2034ac (0xb4ad bytes) 000304[0] 1746146946.300171 trunc from 0x71df08 to 0x269948 (0x4b45bf bytes) 000305[1] 1746146946.339970 mapwrite 0x34c000 thru 0x34f72c (0x372d bytes) 000307[0] 1746146946.461488 trunc from 0x34f72d to 0x18caa2 (0x1c2c8a bytes) 000308[1] 1746146946.481019 mapread 0x084000 thru 0x086b98 (0x2b99 bytes) 000312[1] 1746146946.530142 mapread 0x0b1000 thru 0x0b5c8d (0x4c8e bytes) 000313[1] 1746146946.542447 read_OD 0x141000 thru 0x148fff (0x8000 bytes) 000316[0] 1746146946.552291 write_OD 0x69b000 thru 0x6a2fff (0x8000 bytes) 000318[1] 1746146946.625186 trunc from 0x6a3000 to 0x76b2fb (0xc82fc bytes) 000319[1] 1746146946.664342 mapread 0x43e000 thru 0x44ce8d (0xee8e bytes) 000320[1] 1746146946.754449 write_OD 0x40e000 thru 0x41cfff (0xf000 bytes) 000321[1] 1746146946.827424 mapread 0x5b2000 thru 0x5b3a29 (0x1a2a bytes) 000322[0] 1746146946.851286 write_OD 0x3f7000 thru 0x3fffff (0x9000 bytes) 000324[0] 1746146946.910995 mapread 0x072000 thru 0x07b314 (0x9315 bytes) 000326[0] 1746146946.943074 mapread 0x2e4000 thru 0x2e4240 (0x241 bytes) 000328[1] 1746146946.949733 read_OD 0x062000 thru 0x065fff (0x4000 bytes) 000329[1] 1746146946.953082 write_OD 0x1b6000 thru 0x1c3fff (0xe000 bytes) 000330[1] 1746146946.991441 mapwrite 0x6bc000 thru 0x6c156d (0x556e bytes) 000333[1] 1746146947.075019 trunc from 0x76b2fc to 0x2c722b (0x4a40d0 bytes) 000334[0] 1746146947.103986 trunc from 0x2c722c to 0x68c9ae (0x3c5783 bytes) 000336[1] 1746146947.160342 write_OD 0x8ab000 thru 0x8b5fff (0xb000 bytes) 000337[1] 1746146947.225327 trunc from 0x8b6000 to 0x10e016 (0x7a7fe9 bytes) 000338[1] 1746146947.257539 read_OD 0x0f2000 thru 0x0f7fff (0x6000 bytes) 000339[1] 1746146947.274203 trunc from 0x10e017 to 0x7dd186 (0x6cf170 bytes) 000341[0] 1746146947.330679 mapread 0x7b8000 thru 0x7bd410 (0x5411 bytes) 000342[1] 1746146947.369896 write_OD 0x20b000 thru 0x217fff (0xd000 bytes) 000343[0] 1746146947.445746 mapwrite 0x7f7000 thru 0x7f76f2 (0x6f3 bytes) 000347[0] 1746146947.542238 write_OD 0x1c2000 thru 0x1c2fff (0x1000 bytes) 000351[1] 1746146947.606349 mapread 0x1aa000 thru 0x1b5fc7 (0xbfc8 bytes) 000352[1] 1746146947.664432 mapwrite 0x687000 thru 0x6942ab (0xd2ac bytes) 000354[0] 1746146947.812421 mapwrite 0x1ff000 thru 0x2038f8 (0x48f9 bytes) 000355[1] 1746146947.916480 mapwrite 0x679000 thru 0x680396 (0x7397 bytes) 000357[1] 1746146948.037661 read_OD 0x5b2000 thru 0x5bbfff (0xa000 bytes) 000358[1] 1746146948.050510 read_OD 0x210000 thru 0x21dfff (0xe000 bytes) 000359[0] 1746146948.060571 read_OD 0x21e000 thru 0x222fff (0x5000 bytes) 000361[0] 1746146948.073595 read_OD 0x7dd000 thru 0x7e9fff (0xd000 bytes) 000363[0] 1746146948.079776 trunc from 0x7f76f3 to 0x1e3e43 (0x6138af bytes) 000364[1] 1746146948.110633 mapwrite 0x874000 thru 0x8744de (0x4df bytes) 000365[0] 1746146948.199295 mapread 0x526000 thru 0x5315fb (0xb5fc bytes) 000366[1] 1746146948.243330 mapread 0x40f000 thru 0x41d3d8 (0xe3d9 bytes) 000369[0] 1746146948.271604 mapwrite 0x91d000 thru 0x925c83 (0x8c84 bytes) 000373[1] 1746146948.411957 write_OD 0x77a000 thru 0x780fff (0x7000 bytes) 000375[0] 1746146948.473777 trunc from 0x925c84 to 0x954750 (0x2eacd bytes) 000377[0] 1746146948.561817 mapread 0x114000 thru 0x1160b8 (0x20b9 bytes) 000381[0] 1746146948.605575 trunc from 0x954751 to 0x6cd544 (0x28720c bytes) 000382[1] 1746146948.638718 trunc from 0x6cd545 to 0x045a0d (0x687b37 bytes) 000383[0] 1746146948.653154 write_OD 0x54e000 thru 0x557fff (0xa000 bytes) 000384[0] 1746146948.723264 mapread 0x09f000 thru 0x0abda4 (0xcda5 bytes) 000387[0] 1746146948.752820 write_OD 0x521000 thru 0x529fff (0x9000 bytes) 000390[1] 1746146948.792500 mapwrite 0x066000 thru 0x06ee9d (0x8e9e bytes) 000391[0] 1746146948.867802 mapwrite 0x6f2000 thru 0x6fa723 (0x8724 bytes) 000392[0] 1746146948.988835 trunc from 0x6fa724 to 0x181e30 (0x5788f3 bytes) 000394[1] 1746146949.016733 trunc from 0x181e31 to 0x3885ea (0x2067ba bytes) 000396[1] 1746146949.055427 write_OD 0x1c6000 thru 0x1cffff (0xa000 bytes) 000397[1] 1746146949.103475 mapwrite 0x2d1000 thru 0x2d3acd (0x2ace bytes) 000398[1] 1746146949.184966 mapwrite 0x8fb000 thru 0x8fd75f (0x2760 bytes) 000399[1] 1746146949.302552 trunc from 0x8fd760 to 0x0ce9ae (0x82edb1 bytes) 000401[0] 1746146949.338801 write_OD 0x0ff000 thru 0x106fff (0x8000 bytes) 000402[1] 1746146949.406533 read_OD 0x015000 thru 0x021fff (0xd000 bytes) 000403[1] 1746146949.429669 mapread 0x037000 thru 0x0437b2 (0xc7b3 bytes) 000404[0] 1746146949.468758 read_OD 0x04e000 thru 0x04efff (0x1000 bytes) 000406[1] 1746146949.472156 trunc from 0x107000 to 0x51b83c (0x41483d bytes) 000409[1] 1746146949.515421 mapread 0x47c000 thru 0x48b6ab (0xf6ac bytes) 000411[0] 1746146949.581097 mapwrite 0x060000 thru 0x06bfa7 (0xbfa8 bytes) 000412[1] 1746146949.724270 trunc from 0x51b83d to 0x75c3c1 (0x240b85 bytes) 000414[1] 1746146949.766509 mapread 0x4b2000 thru 0x4c1ccb (0xfccc bytes) 000416[0] 1746146949.830097 mapread 0x4b7000 thru 0x4c5f55 (0xef56 bytes) 000418[1] 1746146949.891458 trunc from 0x75c3c2 to 0x816261 (0xb9ea0 bytes) 000419[1] 1746146949.932029 read_OD 0x565000 thru 0x56ffff (0xb000 bytes) 000420[1] 1746146949.959392 write_OD 0x846000 thru 0x84ffff (0xa000 bytes) 000422[0] 1746146949.999991 mapread 0x450000 thru 0x45645b (0x645c bytes) 000424[1] 1746146950.024798 trunc from 0x850000 to 0x91afa8 (0xcafa9 bytes) 000426[1] 1746146950.063230 mapwrite 0x286000 thru 0x28b20a (0x520b bytes) 000427[1] 1746146950.174887 write_OD 0x302000 thru 0x303fff (0x2000 bytes) 000432[1] 1746146950.252327 mapread 0x71b000 thru 0x7270e1 (0xc0e2 bytes) 000433[0] 1746146950.284755 read_OD 0x1a6000 thru 0x1acfff (0x7000 bytes) 000437[1] 1746146950.291975 write_OD 0x95b000 thru 0x963fff (0x9000 bytes) 000440[0] 1746146950.365203 trunc from 0x964000 to 0x6789ff (0x2eb600 bytes) 000442[0] 1746146950.415327 mapwrite 0x6e1000 thru 0x6e4b37 (0x3b38 bytes) 000445[1] 1746146950.518512 write_OD 0x62d000 thru 0x63afff (0xe000 bytes) 000446[0] 1746146950.576796 mapwrite 0x60a000 thru 0x60da62 (0x3a63 bytes) 000447[1] 1746146950.626268 trunc from 0x6e4b38 to 0x77b930 (0x96df9 bytes) 000451[0] 1746146950.704092 read_OD 0x32f000 thru 0x331fff (0x3000 bytes) 000452[0] 1746146950.727540 mapwrite 0x9ab000 thru 0x9b44d8 (0x94d9 bytes) 000454[0] 1746146950.854084 mapread 00000000 thru 0x005ce3 (0x5ce4 bytes) 000455[0] 1746146950.877052 write_OD 0x7d6000 thru 0x7e2fff (0xd000 bytes) 000457[1] 1746146950.944473 trunc from 0x9b44d9 to 0x699d70 (0x31a768 bytes) 000458[1] 1746146950.982728 write_OD 0x7f3000 thru 0x7fefff (0xc000 bytes) 000459[0] 1746146951.068191 mapwrite 0x8a4000 thru 0x8b2c44 (0xec45 bytes) 000461[0] 1746146951.182458 write_OD 0x881000 thru 0x887fff (0x7000 bytes) 000465[0] 1746146951.239450 read_OD 0x87e000 thru 0x888fff (0xb000 bytes) 000471[0] 1746146951.248821 write_OD 0x7c3000 thru 0x7ccfff (0xa000 bytes) 000472[1] 1746146951.323438 trunc from 0x8b2c45 to 0x5c93e9 (0x2e985b bytes) 000474[0] 1746146951.349025 trunc from 0x5c93ea to 0x4f38ce (0xd5b1b bytes) 000476[1] 1746146951.379690 mapread 0x2a1000 thru 0x2ad3e4 (0xc3e5 bytes) 000480[0] 1746146951.408210 mapwrite 0x5af000 thru 0x5b81ce (0x91cf bytes) 000482[1] 1746146951.496589 write_OD 0x261000 thru 0x265fff (0x5000 bytes) 000488[1] 1746146951.552061 mapwrite 0x004000 thru 0x00af93 (0x6f94 bytes) 000489[0] 1746146951.649520 write_OD 0x07d000 thru 0x08afff (0xe000 bytes) 000491[1] 1746146951.718145 mapwrite 0x26a000 thru 0x26d3bc (0x33bd bytes) 000495[1] 1746146951.854012 trunc from 0x5b81cf to 0x873019 (0x2bae4b bytes) 000497[1] 1746146951.875779 trunc from 0x87301a to 0x379522 (0x4f9af7 bytes) All operations completed A-OK! PASS 16a (36s) == sanityn test 16b: 500 iterations of dual-mount fsx at small size ========================================================== 20:49:14 (1746146954) fallocate on zfs doesn't consume space fallocate not supported 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.432485 s, 19.4 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.442758 s, 18.9 MB/s Chance of close/open is 1 in 50 Seed set to 6956 fd 0: /mnt/lustre/f16b.sanityn fd 1: /mnt/lustre2/f16b.sanityn fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported 000001[0] 1746146957.323947 mapwrite 0x07516d thru 0x076456 (0x12ea bytes) 000002[0] 1746146957.516825 mapread 0x020426 thru 0x021892 (0x146d bytes) truncating to largest ever: 0x30c4ac 000005[0] 1746146957.566258 trunc from 0x076457 to 0x30c4ab (0x296055 bytes) 000006[0] 1746146957.605978 mapwrite 0x905ce4 thru 0x907833 (0x1b50 bytes) 000009[0] 1746146957.737407 mapwrite 0x617972 thru 0x618c1c (0x12ab bytes) 000010[1] 1746146957.824489 read 0x7560dc thru 0x757900 (0x1825 bytes) 000011[0] 1746146957.842027 read 0x8f0ef9 thru 0x8f1a9f (0xba7 bytes) 000012[0] 1746146957.843652 mapread 0x87ec49 thru 0x880a9a (0x1e52 bytes) 000020[1] 1746146957.879280 read 0x5ba311 thru 0x5baad0 (0x7c0 bytes) 000021[0] 1746146957.894443 mapwrite 0x163564 thru 0x1643e2 (0xe7f bytes) 000022[0] 1746146957.950794 write 0x73d8e1 thru 0x73dece (0x5ee bytes) 000024[1] 1746146957.963463 read 0x24de5e thru 0x24ebfd (0xda0 bytes) 000025[1] 1746146957.977685 mapread 0x35f1e4 thru 0x3601a9 (0xfc6 bytes) 000026[0] 1746146958.000441 mapread 0x70fe86 thru 0x710645 (0x7c0 bytes) 000028[1] 1746146958.017398 write 0x7b5b4e thru 0x7b7b1a (0x1fcd bytes) truncating to largest ever: 0x8c6d2e 000033[1] 1746146958.106132 trunc from 0x907834 to 0x8c6d2d (0x40b06 bytes) 000036[0] 1746146958.170917 mapread 0x0765e3 thru 0x07830f (0x1d2d bytes) 000037[0] 1746146958.214154 trunc from 0x8c6d2e to 0x3a6e91 (0x51fe9c bytes) 000039[0] 1746146958.271519 read 0x02abab thru 0x02bd9e (0x11f4 bytes) 000040[1] 1746146958.329436 read 0x2c8552 thru 0x2c9d4f (0x17fe bytes) 000041[0] 1746146958.396168 mapwrite 0x836164 thru 0x837c64 (0x1b01 bytes) 000044[0] 1746146958.573614 mapread 0x14bd1f thru 0x14d3b6 (0x1698 bytes) 000045[1] 1746146958.605406 read 0x479a18 thru 0x47a5e2 (0xbcb bytes) 000047[0] 1746146958.695985 mapwrite 0x238e72 thru 0x23a8e3 (0x1a72 bytes) 000050[0] 1746146958.775867 mapread 0x72503d thru 0x725167 (0x12b bytes) 000051[0] 1746146958.778499 read 0x0b6aea thru 0x0b7b0a (0x1021 bytes) 000052[1] 1746146958.795450 trunc from 0x837c65 to 0x3045a6 (0x5336be bytes) 000053[0] 1746146958.833054 mapwrite 0x056b5f thru 0x0583ed (0x188f bytes) 000057[1] 1746146958.976286 mapread 0x2d5255 thru 0x2d53eb (0x197 bytes) 000059[0] 1746146959.038143 mapread 0x262512 thru 0x262918 (0x407 bytes) 000062[0] 1746146959.084167 mapwrite 0x67d017 thru 0x67ef45 (0x1f2f bytes) 000067[1] 1746146959.320814 write 0x1f108a thru 0x1f2ee8 (0x1e5f bytes) 000069[1] 1746146959.358152 mapread 0x0a4d0f thru 0x0a5b2a (0xe1c bytes) 000070[0] 1746146959.376284 mapread 0x395f4e thru 0x396252 (0x305 bytes) 000071[0] 1746146959.422639 mapread 0x3f9dc8 thru 0x3fafc8 (0x1201 bytes) 000074[1] 1746146959.494000 read 0x08001f thru 0x08066b (0x64d bytes) 000077[1] 1746146959.510246 read 0x3a069f thru 0x3a0c7b (0x5dd bytes) 000080[1] 1746146959.537041 write 0x8edc57 thru 0x8ef5c8 (0x1972 bytes) 000082[1] 1746146959.565043 write 0x380598 thru 0x381aed (0x1556 bytes) 000083[1] 1746146959.591796 trunc from 0x8ef5c9 to 0x53afeb (0x3b45dd bytes) 000084[1] 1746146959.621584 mapwrite 0x85e851 thru 0x85f61a (0xdca bytes) 000086[1] 1746146959.724506 mapread 0x7a13f4 thru 0x7a1af5 (0x702 bytes) 000090[0] 1746146959.764859 mapread 0x5a8fcc thru 0x5a9b56 (0xb8b bytes) 000093[1] 1746146959.806888 trunc from 0x85f61b to 0x58974b (0x2d5ecf bytes) 000094[1] 1746146959.863875 read 0x3b23cd thru 0x3b24a7 (0xdb bytes) 000095[1] 1746146959.890301 write 0x6815af thru 0x681815 (0x267 bytes) 000096[1] 1746146959.897851 trunc from 0x681816 to 0x655be5 (0x2bc30 bytes) 000097[0] 1746146959.913762 read 0x4d8d67 thru 0x4d95d8 (0x872 bytes) 000098[0] 1746146959.987396 write 0x20b9e9 thru 0x20c094 (0x6ac bytes) 000099[1] 1746146960.005523 read 0x1cd3b2 thru 0x1cef50 (0x1b9f bytes) 000102[1] 1746146960.046832 mapread 0x3f698b thru 0x3f842c (0x1aa2 bytes) 000105[0] 1746146960.077995 read 0x029b6e thru 0x029bcc (0x5f bytes) 000106[1] 1746146960.088428 mapread 0x199507 thru 0x19ac30 (0x172a bytes) 000107[1] 1746146960.108986 mapwrite 0x4ca863 thru 0x4cb6e3 (0xe81 bytes) 000108[0] 1746146960.201023 read 0x383cc9 thru 0x384fa5 (0x12dd bytes) 000109[1] 1746146960.205839 trunc from 0x655be6 to 0x7a9088 (0x1534a3 bytes) 000110[0] 1746146960.250872 write 0x912671 thru 0x914490 (0x1e20 bytes) 000112[1] 1746146960.306095 mapread 0x51eabc thru 0x520312 (0x1857 bytes) 000114[0] 1746146960.371225 read 0x0db3a5 thru 0x0db714 (0x370 bytes) 000115[1] 1746146960.408341 read 0x186b62 thru 0x1888cc (0x1d6b bytes) 000116[1] 1746146960.437496 mapread 0x87c864 thru 0x87d9e5 (0x1182 bytes) 000117[0] 1746146960.451696 mapwrite 0x5c92b6 thru 0x5c9de6 (0xb31 bytes) 000118[1] 1746146960.553000 mapread 0x645a33 thru 0x6475e1 (0x1baf bytes) 000119[0] 1746146960.582716 write 0x6ec352 thru 0x6ec9e7 (0x696 bytes) 000124[1] 1746146960.645145 mapread 0x0dad64 thru 0x0dc21b (0x14b8 bytes) truncating to largest ever: 0x95e37a 000126[1] 1746146960.695066 trunc from 0x914491 to 0x95e379 (0x49ee9 bytes) 000128[1] 1746146960.761863 read 0x2e153f thru 0x2e2787 (0x1249 bytes) 000134[0] 1746146960.768488 mapwrite 0x129af7 thru 0x12b652 (0x1b5c bytes) 000142[1] 1746146960.974849 mapread 0x2a496c thru 0x2a4c65 (0x2fa bytes) 000143[1] 1746146961.009242 read 0x16ce1b thru 0x16ce6f (0x55 bytes) 000145[1] 1746146961.012696 mapwrite 0x452371 thru 0x452693 (0x323 bytes) 000148[0] 1746146961.097564 read 0x4b5600 thru 0x4b6f0a (0x190b bytes) 000150[1] 1746146961.118100 mapread 0x37a24e thru 0x37ba48 (0x17fb bytes) 000151[1] 1746146961.136203 write 0x9c2023 thru 0x9c240b (0x3e9 bytes) 000152[1] 1746146961.148862 write 0x0d3e9f thru 0x0d4324 (0x486 bytes) 000155[1] 1746146961.169027 write 0x34af18 thru 0x34c7ff (0x18e8 bytes) 000161[0] 1746146961.238548 trunc from 0x9c240c to 0x734d7f (0x28d68c bytes) 000163[0] 1746146961.329200 write 0x91966c thru 0x91990a (0x29f bytes) 000166[1] 1746146961.373505 read 0x666842 thru 0x6671fc (0x9bb bytes) 000169[1] 1746146961.411537 read 0x5cfcee thru 0x5cff43 (0x256 bytes) 000170[1] 1746146961.421965 read 0x82aa91 thru 0x82ad85 (0x2f5 bytes) 000171[0] 1746146961.438862 write 0x70266d thru 0x704245 (0x1bd9 bytes) 000172[0] 1746146961.465785 write 0x6250d6 thru 0x625835 (0x760 bytes) 000173[1] 1746146961.474324 mapread 0x22b354 thru 0x22cba1 (0x184e bytes) 000174[0] 1746146961.493883 trunc from 0x91990b to 0x3ae1de (0x56b72c bytes) 000175[0] 1746146961.520093 mapread 0x08d7e2 thru 0x08dde7 (0x606 bytes) 000176[0] 1746146961.547040 read 0x33cde1 thru 0x33e9a0 (0x1bc0 bytes) 000177[0] 1746146961.584337 mapwrite 0x190294 thru 0x191b82 (0x18ef bytes) 000178[1] 1746146961.641028 write 0x640da4 thru 0x640ff7 (0x254 bytes) 000181[1] 1746146961.693179 mapwrite 0x89c677 thru 0x89d2ba (0xc44 bytes) 000183[0] 1746146961.799513 mapwrite 0x8364e1 thru 0x836cac (0x7cc bytes) 000187[0] 1746146961.894841 read 0x0ccf36 thru 0x0cee92 (0x1f5d bytes) 000191[0] 1746146961.978877 write 0x1a4d36 thru 0x1a5860 (0xb2b bytes) 000192[0] 1746146962.002554 mapread 0x52e493 thru 0x52f01f (0xb8d bytes) 000193[0] 1746146962.046643 mapread 0x805d42 thru 0x807a29 (0x1ce8 bytes) 000197[0] 1746146962.089216 mapwrite 0x50da8a thru 0x50e221 (0x798 bytes) 000199[1] 1746146962.203025 mapread 0x84d4c9 thru 0x84f02e (0x1b66 bytes) 000200[0] 1746146962.291590 trunc from 0x89d2bb to 0x7ce4c5 (0xcedf5 bytes) 000201[0] 1746146962.316338 mapwrite 0x595f03 thru 0x59649e (0x59c bytes) 000204[0] 1746146962.414177 read 0x0ad8d1 thru 0x0aec0e (0x133e bytes) 000205[1] 1746146962.422931 mapwrite 0x31fe3f thru 0x320ab7 (0xc79 bytes) 000208[0] 1746146962.566205 mapread 0x428f4b thru 0x429a88 (0xb3e bytes) 000210[0] 1746146962.621460 read 0x4ee897 thru 0x4f07a1 (0x1f0b bytes) 000212[1] 1746146962.663975 mapread 0x1a2bc1 thru 0x1a3402 (0x842 bytes) 000213[1] 1746146962.695793 mapread 0x20728d thru 0x20870b (0x147f bytes) 000214[0] 1746146962.724720 write 0x47b078 thru 0x47c7f3 (0x177c bytes) 000215[1] 1746146962.766687 trunc from 0x7ce4c6 to 0x1344bb (0x69a00a bytes) 000220[0] 1746146962.860447 mapread 0x0aab99 thru 0x0ab561 (0x9c9 bytes) 000221[0] 1746146962.906819 mapread 0x12cb4f thru 0x12d86f (0xd21 bytes) 000222[1] 1746146962.924776 write 0x19acef thru 0x19b7ff (0xb11 bytes) 000223[1] 1746146962.956873 mapread 0x05a704 thru 0x05b3e0 (0xcdd bytes) 000226[0] 1746146963.035898 trunc from 0x19b800 to 0x077678 (0x124187 bytes) 000228[1] 1746146963.080496 write 0x1f307a thru 0x1f48cf (0x1856 bytes) 000229[0] 1746146963.109976 mapread 0x0d7f57 thru 0x0d96bf (0x1769 bytes) 000236[0] 1746146963.204517 mapwrite 0x5c5ab6 thru 0x5c66fc (0xc47 bytes) 000237[0] 1746146963.274493 read 0x0f6be4 thru 0x0f72d8 (0x6f5 bytes) 000239[0] 1746146963.322438 read 0x51b033 thru 0x51ba51 (0xa1f bytes) 000241[1] 1746146963.349010 mapread 0x19f70d thru 0x1a1365 (0x1c59 bytes) 000242[0] 1746146963.416598 mapwrite 0x88a44c thru 0x88bddb (0x1990 bytes) 000244[0] 1746146963.570322 read 0x6b68ce thru 0x6b7d0f (0x1442 bytes) 000245[1] 1746146963.595220 mapread 0x31e5dd thru 0x31eb40 (0x564 bytes) 000246[0] 1746146963.618819 mapwrite 0x6feffa thru 0x700a10 (0x1a17 bytes) 000247[1] 1746146963.737562 read 0x1ecb57 thru 0x1ed15f (0x609 bytes) 000250[1] 1746146963.769926 mapread 0x50129e thru 0x501a11 (0x774 bytes) 000251[0] 1746146963.797598 write 0x845404 thru 0x846b4b (0x1748 bytes) 000252[0] 1746146963.821351 write 0x89b94d thru 0x89ba2e (0xe2 bytes) 000253[0] 1746146963.845472 write 0x59abfd thru 0x59bdf9 (0x11fd bytes) 000255[1] 1746146963.902196 read 0x1fd78a thru 0x1fe4d7 (0xd4e bytes) 000257[1] 1746146963.950861 trunc from 0x89ba2f to 0x10a0dd (0x791951 bytes) 000259[0] 1746146963.986761 read 0x02f349 thru 0x03073f (0x13f7 bytes) 000261[0] 1746146964.028273 trunc from 0x10a0de to 0x4ff081 (0x3f4fa4 bytes) 000264[0] 1746146964.083731 mapwrite 0x807076 thru 0x807a06 (0x991 bytes) 000265[0] 1746146964.184654 write 0x549ff3 thru 0x54a803 (0x811 bytes) 000266[0] 1746146964.203940 mapwrite 0x795866 thru 0x79782d (0x1fc8 bytes) 000270[0] 1746146964.299988 read 0x32255c thru 0x322702 (0x1a7 bytes) 000271[0] 1746146964.321875 trunc from 0x807a07 to 0x57dcae (0x289d58 bytes) 000272[0] 1746146964.366328 mapwrite 0x213dd0 thru 0x214a14 (0xc45 bytes) 000276[1] 1746146964.437266 trunc from 0x57dcaf to 0x3cf906 (0x1ae3a8 bytes) 000280[0] 1746146964.515036 read 0x14369a thru 0x144950 (0x12b7 bytes) 000281[0] 1746146964.590822 read 0x07ee9f thru 0x07fd2b (0xe8d bytes) 000283[0] 1746146964.609450 trunc from 0x3cf907 to 0x2f4eaf (0xdaa57 bytes) 000286[1] 1746146964.657502 mapread 0x11f118 thru 0x11f4d9 (0x3c2 bytes) 000287[1] 1746146964.713519 mapread 0x01646f thru 0x01808b (0x1c1d bytes) 000289[0] 1746146964.775672 mapwrite 0x3f67da thru 0x3f71df (0xa06 bytes) 000290[1] 1746146964.939023 trunc from 0x3f71e0 to 0x824976 (0x42d797 bytes) 000291[1] 1746146964.999134 read 0x46bd10 thru 0x46dcac (0x1f9d bytes) 000293[0] 1746146965.030189 write 0x2f7738 thru 0x2f80cf (0x998 bytes) 000294[1] 1746146965.069510 mapread 0x4c2a83 thru 0x4c2ae1 (0x5f bytes) 000298[0] 1746146965.116322 mapwrite 0x9b2452 thru 0x9b4161 (0x1d10 bytes) 000299[0] 1746146965.260705 write 0x2ba1f6 thru 0x2bac54 (0xa5f bytes) 000300[0] 1746146965.263900 write 0x25cd73 thru 0x25eaf5 (0x1d83 bytes) 000302[1] 1746146965.286771 mapread 0x26a18c thru 0x26b588 (0x13fd bytes) 000305[0] 1746146965.352736 write 0x46e57f thru 0x47056f (0x1ff1 bytes) 000306[1] 1746146965.364927 write 0x8d1bfc thru 0x8d25c6 (0x9cb bytes) 000307[1] 1746146965.410185 mapwrite 0x2b96d7 thru 0x2b978e (0xb8 bytes) 000308[0] 1746146965.541499 read 0x0b88f6 thru 0x0b95c1 (0xccc bytes) 000312[0] 1746146965.573397 write 0x88bc4b thru 0x88c776 (0xb2c bytes) 000316[1] 1746146965.606312 trunc from 0x9b4162 to 0x56a6cd (0x449a94 bytes) 000318[0] 1746146965.682827 trunc from 0x56a6ce to 0x2a4d58 (0x2c5975 bytes) 000322[0] 1746146965.791211 read 0x0b3f4f thru 0x0b4d9b (0xe4d bytes) 000323[1] 1746146965.841423 mapread 0x1212ef thru 0x122e80 (0x1b92 bytes) 000328[1] 1746146965.940599 trunc from 0x2a4d59 to 0x4c8328 (0x2235d0 bytes) 000331[0] 1746146966.020902 mapwrite 0x3798bf thru 0x37af99 (0x16db bytes) 000336[1] 1746146966.231823 trunc from 0x4c8329 to 0x80f823 (0x3474fb bytes) 000337[0] 1746146966.263578 write 0x687d63 thru 0x6899b1 (0x1c4f bytes) 000338[1] 1746146966.365859 trunc from 0x80f824 to 0x39047e (0x47f3a5 bytes) 000343[1] 1746146966.503066 mapread 0x285422 thru 0x2867d9 (0x13b8 bytes) 000345[0] 1746146966.539537 mapwrite 0x6f68ee thru 0x6f692b (0x3e bytes) 000346[0] 1746146966.655304 trunc from 0x6f692c to 0x75697a (0x6004f bytes) 000351[0] 1746146966.754173 read 0x498bbc thru 0x49a7cf (0x1c14 bytes) 000355[1] 1746146966.802020 write 0x13c8e3 thru 0x13dbc4 (0x12e2 bytes) 000356[0] 1746146966.832001 mapwrite 0x4a4eaa thru 0x4a5778 (0x8cf bytes) 000357[0] 1746146966.930834 mapread 0x3fb734 thru 0x3fbb60 (0x42d bytes) 000361[0] 1746146966.995809 trunc from 0x75697b to 0x8f7316 (0x1a099c bytes) 000364[0] 1746146967.033894 trunc from 0x8f7317 to 0x012df2 (0x8e4524 bytes) 000368[1] 1746146967.088660 write 0x157fca thru 0x1587c1 (0x7f8 bytes) 000369[0] 1746146967.126394 write 0x6351dc thru 0x636a98 (0x18bd bytes) 000370[0] 1746146967.142042 write 0x312b46 thru 0x31460c (0x1ac7 bytes) 000371[0] 1746146967.165246 write 0x62ebd2 thru 0x63000e (0x143d bytes) 000373[1] 1746146967.205007 trunc from 0x636a99 to 0x2f71ac (0x33f8ec bytes) 000374[0] 1746146967.269536 mapwrite 0x9a7a71 thru 0x9a82ba (0x84a bytes) 000382[0] 1746146967.520546 write 0x384de1 thru 0x3863e3 (0x1603 bytes) truncating to largest ever: 0x9fb1d8 000384[0] 1746146967.552403 trunc from 0x9a82bb to 0x9fb1d7 (0x52f1d bytes) 000386[0] 1746146967.578252 write 0x8d24af thru 0x8d2ffb (0xb4d bytes) 000388[0] 1746146967.664734 read 0x7586dd thru 0x758fdc (0x900 bytes) 000389[1] 1746146967.677201 trunc from 0x9fb1d8 to 0x33667a (0x6c4b5d bytes) 000392[0] 1746146967.747669 write 0x251496 thru 0x252567 (0x10d2 bytes) 000394[0] 1746146967.795920 trunc from 0x33667b to 0x1c0909 (0x175d71 bytes) 000395[0] 1746146967.849722 mapread 0x04fb69 thru 0x05180b (0x1ca3 bytes) 000398[0] 1746146967.935229 trunc from 0x1c090a to 0x5513f5 (0x390aec bytes) 000399[0] 1746146967.969278 mapread 0x3058ee thru 0x305c04 (0x317 bytes) 000400[1] 1746146967.978338 mapread 0x277f23 thru 0x278a29 (0xb07 bytes) 000402[1] 1746146968.072571 trunc from 0x5513f6 to 0x7a55b1 (0x2541bc bytes) 000404[0] 1746146968.162956 write 0x79a982 thru 0x79a9c0 (0x3f bytes) 000405[0] 1746146968.195827 read 0x42d418 thru 0x42e034 (0xc1d bytes) 000406[0] 1746146968.226575 write 0x91222b thru 0x91280c (0x5e2 bytes) 000410[1] 1746146968.260813 read 0x61964e thru 0x61a240 (0xbf3 bytes) 000411[1] 1746146968.294315 read 0x0d3d04 thru 0x0d5b96 (0x1e93 bytes) 000412[0] 1746146968.324272 mapread 0x6c93bf thru 0x6cae66 (0x1aa8 bytes) 000414[1] 1746146968.333249 read 0x60ba1a thru 0x60cf79 (0x1560 bytes) 000418[1] 1746146968.338058 write 0x52cef3 thru 0x52e692 (0x17a0 bytes) 000419[1] 1746146968.367489 read 0x72a9c7 thru 0x72c52d (0x1b67 bytes) 000421[0] 1746146968.404341 mapwrite 0x9ffe56 thru 0x9fffff (0x1aa bytes) 000424[1] 1746146968.531233 trunc from 0xa00000 to 0x1e2370 (0x81dc8f bytes) 000425[1] 1746146968.583450 write 0x4fa6d0 thru 0x4fb4fb (0xe2c bytes) 000426[0] 1746146968.585162 mapwrite 0x8d1364 thru 0x8d13bb (0x58 bytes) 000427[1] 1746146968.738617 read 0x0425ed thru 0x042a85 (0x499 bytes) 000428[1] 1746146968.759072 mapwrite 0x607f7f thru 0x6092d1 (0x1353 bytes) 000433[1] 1746146968.871680 write 0x23a450 thru 0x23ad85 (0x936 bytes) 000434[0] 1746146968.889193 read 0x020f1e thru 0x0222e5 (0x13c8 bytes) 000435[0] 1746146968.931246 mapread 0x414256 thru 0x416075 (0x1e20 bytes) 000436[1] 1746146968.974256 mapwrite 0x544a5f thru 0x54544f (0x9f1 bytes) 000439[1] 1746146969.081274 read 0x7c2096 thru 0x7c2f02 (0xe6d bytes) 000440[0] 1746146969.114336 read 0x6e4535 thru 0x6e5415 (0xee1 bytes) 000442[1] 1746146969.132806 read 0x13ce23 thru 0x13de21 (0xfff bytes) 000444[1] 1746146969.137723 trunc from 0x8d13bc to 0x37990b (0x557ab0 bytes) 000446[0] 1746146969.184347 read 0x2a1e2d thru 0x2a2da8 (0xf7c bytes) 000449[0] 1746146969.247236 write 0x910f37 thru 0x9114a7 (0x571 bytes) 000451[0] 1746146969.293917 read 0x20007a thru 0x2009f6 (0x97d bytes) 000452[0] 1746146969.313147 mapread 0x197dc9 thru 0x19965c (0x1894 bytes) 000457[1] 1746146969.411744 trunc from 0x9114a8 to 0x023d43 (0x8ed764 bytes) 000458[0] 1746146969.437726 write 0x4aa3a2 thru 0x4aaf4a (0xba9 bytes) 000463[1] 1746146969.514567 write 0x724380 thru 0x72451d (0x19e bytes) 000466[1] 1746146969.524348 mapwrite 0x836238 thru 0x838208 (0x1fd1 bytes) 000467[0] 1746146969.676609 read 0x1fabca thru 0x1fb3fb (0x832 bytes) 000468[0] 1746146969.693226 write 0x8b0efb thru 0x8b2a36 (0x1b3c bytes) 000471[0] 1746146969.738606 mapwrite 0x2d265a thru 0x2d287d (0x224 bytes) 000472[1] 1746146969.812771 mapread 0x822cb5 thru 0x822eff (0x24b bytes) 000475[1] 1746146969.869153 write 0x03774d thru 0x037d3e (0x5f2 bytes) 000476[1] 1746146969.900435 write 0x683858 thru 0x6847fb (0xfa4 bytes) 000479[0] 1746146969.943201 mapread 0x5dcd28 thru 0x5de2d8 (0x15b1 bytes) 000481[0] 1746146969.980244 mapwrite 0x417380 thru 0x417f2e (0xbaf bytes) 000482[0] 1746146970.058183 mapwrite 0x607b08 thru 0x609341 (0x183a bytes) 000483[0] 1746146970.148426 mapread 0x844261 thru 0x844528 (0x2c8 bytes) 000484[1] 1746146970.177699 trunc from 0x8b2a37 to 0x87514e (0x3d8e8 bytes) 000488[0] 1746146970.213669 read 0x27d76a thru 0x27f1db (0x1a72 bytes) 000492[1] 1746146970.262613 write 0x5c235c thru 0x5c25b1 (0x256 bytes) 000494[0] 1746146970.292133 write 0x383063 thru 0x384dee (0x1d8c bytes) 000497[1] 1746146970.344789 mapwrite 0x92b1bd thru 0x92b4f9 (0x33d bytes) 000498[0] 1746146970.422093 mapread 0x34ec9c thru 0x34f75f (0xac4 bytes) All operations completed A-OK! PASS 16b (18s) == sanityn test 16c: verify data consistency on ldiskfs with cache disabled (b=17397) ========================================================== 20:49:32 (1746146972) SKIP: sanityn test_16c dio on ldiskfs only SKIP 16c (3s) == sanityn test 16d: Verify DIO and buffer IO with two clients ========================================================== 20:49:35 (1746146975) 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.505782 s, 16.6 MB/s 2+0 records in 2+0 records out 8388608 bytes (8.4 MB) copied, 0.342062 s, 24.5 MB/s ldlm.namespaces.MGC192.168.201.120@tcp.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800aa215800.lru_size=clear ldlm.namespaces.lustre-MDT0000-mdc-ffff8800b6e70800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800aa215800.lru_size=clear ldlm.namespaces.lustre-OST0000-osc-ffff8800b6e70800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800aa215800.lru_size=clear ldlm.namespaces.lustre-OST0001-osc-ffff8800b6e70800.lru_size=clear 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.55445 s, 11.0 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 2.20947 s, 47.5 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.66192 s, 63.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 3.85718 s, 27.2 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 6.94589 s, 15.1 MB/s 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 1.9646 s, 53.4 MB/s PASS 16d (34s) == sanityn test 16e: Verify size consistency for O_DIRECT write ========================================================== 20:50:09 (1746147009) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.172173 s, 60.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.122324 s, 8.6 MB/s PASS 16e (5s) == sanityn test 16f: rw sequential consistency vs drop_caches ========================================================== 20:50:14 (1746147014) TIMEOUT PASS 16f (24s) == sanityn test 16g: mmap rw sequential consistency vs drop_caches ========================================================== 20:50:39 (1746147039) TIMEOUT PASS 16g (24s) == sanityn test 16h: mmap read after truncate file ======= 20:51:04 (1746147064) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16h (5s) == sanityn test 16i: read after truncate file ============ 20:51:09 (1746147069) ------- before truncate -------- 0003220 3035 3a31 3035 3a31 2f3a 6f68 656d 732f 0003240 6e61 7469 7579 7273 3a31 622f 6e69 622f 0003260 7361 0a68 0003264 ------- after truncate -------- 0001460 7373 6468 783a 373a 3a34 3437 503a 6972 0001500 6976 656c 6567 732d 7065 7261 7461 6465 0001520 5320 4853 2f3a 6176 2f72 0001532 PASS 16i (5s) == sanityn test 16j: race dio with buffered i/o ========== 20:51:14 (1746147074) 1+0 records in 1+0 records out 26214400 bytes (26 MB) copied, 0.886282 s, 29.6 MB/s bs: 1044480, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.830227 s, 31.6 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 2.39479 s, 10.9 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1048576, file_size 26214400 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.475114 s, 55.2 MB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 2.08473 s, 12.6 MB/s /mnt/lustre/f16j.sanityn.2 has type file OK /mnt/lustre/f16j.sanityn.2 has size 26214400 OK /mnt/lustre2/f16j.sanityn.2 has type file OK /mnt/lustre2/f16j.sanityn.2 has size 26214400 OK bs: 1049600, file_size 26214400 dd: error reading '/mnt/lustre/f16j.sanityn.1': Invalid argument 0+0 records in 0+0 records out 0 bytes (0 B) copied, 0.146224 s, 0.0 kB/s 12+1 records in 12+1 records out 26214400 bytes (26 MB) copied, 0.425542 s, 61.6 MB/s sanityn test_16j: @@@@@@ FAIL: dio copy 1 w/bsize 1049600 failed: 1 Trace dump: = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7249:error() = /home/green/git/lustre-release/lustre/tests/sanityn.sh:760:test_16j() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7622:run_one() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7685:run_one_logged() = /home/green/git/lustre-release/lustre/tests/test-framework.sh:7503:run_test() = /home/green/git/lustre-release/lustre/tests/sanityn.sh:830:main() Dumping lctl log to /tmp/testlogs//sanityn.test_16j.*.1746147086.log rsync: chown "/tmp/testlogs/.sanityn.test_16j.debug_log.oleg120-server.1746147086.log.AlLdiE" failed: Operation not permitted (1) rsync: chown "/tmp/testlogs/.sanityn.test_16j.dmesg.oleg120-server.1746147086.log.5C4yJF" failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1651) [generator=3.1.2] FAIL 16j (16s) == sanityn test 16k: Parallel FSX and drop caches should not panic ========================================================== 20:51:34 (1746147094) + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.1 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.2 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.3 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.4 -l 67108864 + /home/green/git/lustre-release/lustre/tests/fsx -N 1000 /mnt/lustre/fsxfile.5 -l 67108864 fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read fsx: test_fallocate: filesystem does not support fallocate mode 0x0, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x3, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x10, disabling!: Operation not supported fsx: test_fallocate: filesystem does not support fallocate mode 0x1, disabling!: Operation not supported skipping zero size read truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x1a7062c truncating to largest ever: 0x232e1d3 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x32cce75 truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c569db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3c711db truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f0e56b truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab truncating to largest ever: 0x3f706ab All operations completed A-OK! All operations completed A-OK! + fsx 1 done: rc=0 Thu May 1 20:52:07 EDT 2025 All operations completed A-OK! All operations completed A-OK! + fsx 2 done: rc=0 Thu May 1 20:52:08 EDT 2025 + fsx 3 done: rc=0 Thu May 1 20:52:08 EDT 2025 All operations completed A-OK! + fsx 4 done: rc=0 Thu May 1 20:52:08 EDT 2025 + fsx 5 done: rc=0 Thu May 1 20:52:08 EDT 2025 PASS 16k (39s) == sanityn test 17: resource creation/LVB creation race ========================================================================= 20:52:13 (1746147133) fail_loc=0x8000030a PASS 17 (8s) == sanityn test 18: mmap sanity check =========================================================================================== 20:52:21 (1746147141) mmap test1: basic mmap operation (PASS, 0.10853s) mmap test2: MAP_PRIVATE not write back (PASS, 0.067074s) mmap test3: concurrent mmap ops on two nodes (PASS, 22.958s) mmap test4: c1 write to f1 from mmapped f2, c2 write to f1 from mmapped f1 (PASS, 4.4529s) mmap test5: read/write file to/from the buffer which mmapped to just this file (PASS, 0.18137s) mmap test6: check mmap write/read content on two nodes (PASS, 0.22664s) mmap test7: file i/o with an unmapped buffer (SKIPPED, 0s) mmap test8: SIGBUS for beyond file size (SKIPPED, 0s) mmap test9: SIGBUS for negative file offset (SKIPPED, 0s) mmap test10: multi-thread mmap access (PASS, 0.98323s) mmap test11: multi-thread shared mmap (PASS, 0.87259s) mmap test10: mtime not change for readonly mmap access (PASS, 1.0537s) PASS 18 (36s) == sanityn test 19: test concurrent uncached read races ========================================================================= 20:52:58 (1746147178) oleg120-server: error: get_param: param_path 'osd-*/lustre-OST*/read_cache_enable': No such file or directory pdsh@oleg120-client: oleg120-server: ssh exited with exit code 2 SKIP: sanityn test_19 not cache-capable obdfilter SKIP 19 (3s) == sanityn test 20: test extra readahead page left in cache ============================================================== 20:53:01 (1746147181) PASS 20 (6s) == sanityn test 21: Try to remove mountpoint on another dir ============================================================== 20:53:07 (1746147187) rmdir: removing directory, '/mnt/lustre/d21.sanityn' rmdir: failed to remove '/mnt/lustre/d21.sanityn': Device or resource busy rmdir: removing directory, '/mnt/lustre2/d21.sanityn' Removed mounted directory from another mountpoint, needs to be fixed PASS 21 (5s) == sanityn test 23: others should see updated atime while another read============================================================== 20:53:13 (1746147193) now is 1746147195 starting reads multiop /mnt/lustre/f23.sanityn vor20_c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 new atime is 1746147256 PASS 23 (67s) == sanityn test 24a: lfs df [-ih] [path] test =================================================================================== 20:54:20 (1746147260) UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 19456 3749888 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 45056 3724288 2% /mnt/lustre[OST:1] filesystem_summary: 7542784 64512 7474176 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 19456 3749888 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 45056 3724288 2% /mnt/lustre2[OST:1] filesystem_summary: 7542784 64512 7474176 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 114.9K 404 114.5K 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 114.1K 404 113.7K 1% /mnt/lustre[OST:1] filesystem_summary: 228.6K 387 228.2K 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 114.9K 404 114.5K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.1K 404 113.7K 1% /mnt/lustre2[OST:1] filesystem_summary: 228.6K 387 228.2K 1% /mnt/lustre2 UUID bytes Used Available Use% Mounted on lustre-MDT0000_UUID 2.1G 3.2M 2.1G 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3.6G 19.0M 3.6G 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3.6G 44.0M 3.6G 2% /mnt/lustre[OST:1] filesystem_summary: 7.2G 63.0M 7.1G 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117652 404 117248 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 116852 404 116448 1% /mnt/lustre2[OST:1] filesystem_summary: 234083 387 233696 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 19456 3749888 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 45056 3724288 2% /mnt/lustre[OST:1] filesystem_summary: 7542784 64512 7474176 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 490.6K 387 490.2K 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 114.9K 404 114.5K 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 114.1K 404 113.7K 1% /mnt/lustre2[OST:1] filesystem_summary: 228.6K 387 228.2K 1% /mnt/lustre2 UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre[MDT:0] filesystem_summary: 502378 387 501991 1% /mnt/lustre UUID Inodes IUsed IFree IUse% Mounted on lustre-MDT0000_UUID 502378 387 501991 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 117652 404 117248 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 116852 404 116448 1% /mnt/lustre2[OST:1] filesystem_summary: 234083 387 233696 1% /mnt/lustre2 UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre[MDT:0] lustre-OST0000_UUID 3771392 19456 3749888 1% /mnt/lustre[OST:0] lustre-OST0001_UUID 3771392 45056 3724288 2% /mnt/lustre[OST:1] filesystem_summary: 7542784 64512 7474176 1% /mnt/lustre UUID 1K-blocks Used Available Use% Mounted on lustre-MDT0000_UUID 2210688 3328 2205312 1% /mnt/lustre2[MDT:0] lustre-OST0000_UUID 3771392 19456 3749888 1% /mnt/lustre2[OST:0] lustre-OST0001_UUID 3771392 45056 3724288 2% /mnt/lustre2[OST:1] filesystem_summary: 7542784 64512 7474176 1% /mnt/lustre2 PASS 24a (6s) == sanityn test 24b: lfs df should show both filesystems ========================================================================= 20:54:25 (1746147265) PASS 24b (5s) == sanityn test 25a: change ACL on one mountpoint be seen on another ============================================================= 20:54:30 (1746147270) running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] Can't lstat /mnt/lustre2/d25a.sanityn/f1: Permission denied running as uid/gid/euid/egid 500/500/500/500, groups: 500 [checkstat] [-v] [/mnt/lustre2/d25a.sanityn/f1] PASS 25a (6s) == sanityn test 25b: change ACL under remote dir on one mountpoint be seen on another ========================================================== 20:54:36 (1746147276) SKIP: sanityn test_25b needs >= 2 MDTs SKIP 25b (3s) == sanityn test 26a: allow mtime to get older ============ 20:54:39 (1746147279) utime: good mknod times 1746147280 <= 1746147281 <= 1746147281 for /mnt/lustre/f26a utime: good mknod times 1746147280 <= 1746147281 <= 1746147281 for /mnt/lustre2/f26a utime: good utime mtimes 100000, atime 200000 utime: updated times for /mnt/lustre2/f26a PASS 26a (6s) == sanityn test 26b: sync mtime between ost and mds ====== 20:54:45 (1746147285) PASS 26b (8s) == sanityn test 26c: set-in-past on open file is not lost on close ========================================================== 20:54:53 (1746147293) multiop /mnt/lustre/f26c.sanityn vO_c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 File: '/mnt/lustre/f26c.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502391 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2025-05-01 20:54:54.000000000 -0400 Birth: - File: '/mnt/lustre2/f26c.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205272502391 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2000-12-31 06:12:59.000000000 -0500 Modify: 2000-12-31 06:12:59.000000000 -0500 Change: 2025-05-01 20:54:54.000000000 -0400 Birth: - PASS 26c (6s) SKIP: sanityn test_27 skipping excluded test 27 SKIP: sanityn test_28 skipping ALWAYS excluded test 28 == sanityn test 30: recreate file race =================== 20:55:02 (1746147302) 0 PASS 30 (7s) == sanityn test 31a: voluntary cancel / blocking ast race======================================================================== 20:55:09 (1746147309) fail_loc=0x314 PASS 31a (6s) == sanityn test 31b: voluntary OST cancel / blocking ast race======================================================================== 20:55:15 (1746147315) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg120-server mds-ost sync done. sleep 5 for ZFS MDS Waiting for MDT destroys to complete fail_loc=0x314 fail_loc=0x316 fail_loc=0 fail_loc=0 Filesystem 1K-blocks Used Available Use% Mounted on 192.168.201.120@tcp:/lustre 7542784 64512 7474176 1% /mnt/lustre2 PASS 31b (19s) == sanityn test 31r: open-rename(replace) race =========== 20:55:34 (1746147334) -rw-r--r-- 1 root root 0 May 1 20:55 /mnt/lustre/f31r.sanityn.target fail_loc=0x1419 fail_val=3 PASS 31r (8s) == sanityn test 31s: open should not revalidate invalid dentry ========================================================== 20:55:42 (1746147342) /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 1213: 13715 Terminated tail -f $tfile PASS 31s (6s) == sanityn test 31t: getattr should not revalidate invalid dentry ========================================================== 20:55:49 (1746147349) PASS 31t (6s) SKIP: sanityn test_33a skipping SLOW test 33a == sanityn test 33b: COS: cross create/delete, 2 clients, benchmark under remote dir ========================================================== 20:55:57 (1746147357) SKIP: sanityn test_33b Need two or more clients, have 1 SKIP 33b (3s) == sanityn test 33c: Cancel cross-MDT lock should trigger Sync-on-Lock-Cancel ========================================================== 20:56:00 (1746147360) SKIP: sanityn test_33c needs >= 2 MDTs SKIP 33c (3s) == sanityn test 33d: dependent transactions should trigger COS ========================================================== 20:56:03 (1746147363) SKIP: sanityn test_33d needs >= 2 MDTs SKIP 33d (2s) == sanityn test 33e: independent transactions shouldn't trigger COS ========================================================== 20:56:05 (1746147365) SKIP: sanityn test_33e needs >= 2 MDTs SKIP 33e (3s) == sanityn test 34: no lock timeout under IO ============= 20:56:08 (1746147368) fail_loc=0x512 fail_loc=0x512 lock should not expire writing on client1 reading on client2 success fail_loc=0x511 fail_loc=0x511 lock should expire writing on client1 reading on client2 success oleg120-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa215800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800aa215800.ost_server_uuid in FULL state after 0 sec oleg120-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa215800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800aa215800.ost_server_uuid in IDLE state after 0 sec oleg120-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa215800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800aa215800.ost_server_uuid in FULL state after 0 sec oleg120-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa215800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800aa215800.ost_server_uuid in IDLE state after 0 sec oleg120-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa215800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800aa215800.ost_server_uuid in FULL state after 0 sec oleg120-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8800aa215800.ost_server_uuid 50 osc.lustre-OST0001-osc-ffff8800aa215800.ost_server_uuid in IDLE state after 0 sec PASS 34 (114s) == sanityn test 35: -EINTR cp_ast vs. bl_ast race does not evict client ========================================================== 20:58:02 (1746147482) fail_loc=0x80000317 Race attempt 0 Wait for 20392 20483 for 60 sec... - open/close 951 (time 1746147497.58 total 10.00 last 95.10) - open/close 1886 (time 1746147507.59 total 20.00 last 93.46) - open/close 2724 (time 1746147517.60 total 30.02 last 83.70) - open/close 3585 (time 1746147527.61 total 40.02 last 86.04) total: 4000 open/close in 44.91 seconds: 89.07 ops/second fail_loc=0x0 Filesystem Size Used Avail Use% Mounted on 192.168.201.120@tcp:/lustre 7.2G 64M 7.2G 1% /mnt/lustre 192.168.201.120@tcp:/lustre 7.2G 64M 7.2G 1% /mnt/lustre2 PASS 35 (69s) == sanityn test 36: handle ESTALE/open-unlink correctly == 20:59:11 (1746147551) Waiting for orphan cleanup... osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed wait 40 secs maximumly for oleg120-server mds-ost sync done. Waiting for MDT destroys to complete 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 1.00905 s, 52.0 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(0) *** before(7473152) after_dd(7415808) after(7467008) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 1.07146 s, 48.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(1) *** before(7467008) after_dd(7415808) after(7467008) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.853603 s, 61.4 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(2) *** before(7467008) after_dd(7417856) after(7469056) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.631956 s, 83.0 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(3) *** before(7469056) after_dd(7415808) after(7467008) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.717041 s, 73.1 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(4) *** before(7467008) after_dd(7419904) after(7471104) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.86342 s, 60.7 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(5) *** before(7471104) after_dd(7417856) after(7469056) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.673145 s, 77.9 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(6) *** before(7469056) after_dd(7417856) after(7469056) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.767296 s, 68.3 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 sleep 5 for ZFS MDS Waiting for MDT destroys to complete *** cycle(7) *** before(7469056) after_dd(7417856) after(7469056) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.688215 s, 76.2 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(8) *** before(7469056) after_dd(7415808) after(7467008) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.80388 s, 65.2 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete *** cycle(9) *** before(7467008) after_dd(7410688) after(7461888) 50+0 records in 50+0 records out 52428800 bytes (52 MB) copied, 0.659333 s, 79.5 MB/s multiop /mnt/lustre2/d36.sanityn/f36.sanityn vO_r52428800c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 sleep 5 for ZFS MDS Waiting for MDT destroys to complete *** cycle(10) *** before(7461888) after_dd(7415808) after(7467008) PASS 36 (238s) == sanityn test 37: check i_size is not updated for directory on close (bug 18695) ======================================================================== 21:03:09 (1746147789) multiop /mnt/lustre/d37.sanityn vD_c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 - create 1864 (time 1746147801.89 total 10.00 last 186.35) - create 3856 (time 1746147811.89 total 20.01 last 199.08) - create 5933 (time 1746147821.89 total 30.01 last 207.65) - create 7861 (time 1746147831.89 total 40.01 last 192.78) - create 9796 (time 1746147841.90 total 50.02 last 193.44) total: 10000 create in 51.18 seconds: 195.39 ops/second PASS 37 (58s) == sanityn test 39a: file mtime does not change after rename ========================================================== 21:04:07 (1746147847) repeat after cancel_lru_locks PASS 39a (5s) == sanityn test 39b: file mtime the same on clients with/out lock ========================================================== 21:04:12 (1746147852) repeat after cancel_lru_locks PASS 39b (5s) == sanityn test 39c: check truncate mtime update ================================================================================ 21:04:17 (1746147857) repeat after cancel_lru_locks PASS 39c (5s) == sanityn test 39d: sync write should update mtime ====== 21:04:23 (1746147863) fail_loc=0x411 fail_loc=0 PASS 39d (3s) == sanityn test 40a: pdirops: create vs others ======================================================================== 21:04:26 (1746147866) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40a (14s) == sanityn test 40b: pdirops: open|create and others ======================================================================== 21:04:40 (1746147880) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40b (13s) == sanityn test 40c: pdirops: link and others ======================================================================== 21:04:53 (1746147893) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40c (13s) == sanityn test 40d: pdirops: unlink and others ======================================================================== 21:05:06 (1746147906) No conflict No conflict No conflict No conflict No conflict No conflict No conflict PASS 40d (13s) == sanityn test 40e: pdirops: rename and others ======================================================================== 21:05:19 (1746147919) No conflict No conflict No conflict No conflict No conflict No conflict PASS 40e (12s) == sanityn test 41a: pdirops: create vs mkdir ======================================================================== 21:05:31 (1746147931) mkdir: cannot create directory '/mnt/lustre2/f41a.sanityn': File exists Conflict PASS 41a (8s) == sanityn test 41b: pdirops: create vs create ======================================================================== 21:05:39 (1746147939) open: File exists Conflict PASS 41b (10s) == sanityn test 41c: pdirops: create vs link ======================================================================== 21:05:49 (1746147949) link: cannot create link '/mnt/lustre2/f41c.sanityn' to '/mnt/lustre2/f41c.sanityn-2': File exists Conflict PASS 41c (9s) == sanityn test 41d: pdirops: create vs unlink ======================================================================== 21:05:58 (1746147958) Conflict PASS 41d (11s) == sanityn test 41e: pdirops: create and rename (tgt) ======================================================================== 21:06:09 (1746147969) Conflict PASS 41e (9s) == sanityn test 41f: pdirops: create and rename (src) ======================================================================== 21:06:18 (1746147978) Conflict PASS 41f (9s) == sanityn test 41g: pdirops: create vs getattr ======================================================================== 21:06:27 (1746147987) Conflict PASS 41g (10s) == sanityn test 41h: pdirops: create vs readdir ======================================================================== 21:06:37 (1746147997) Conflict PASS 41h (10s) == sanityn test 41i: reint_open: create vs create ======== 21:06:47 (1746148007) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 41i (910s) == sanityn test 42a: pdirops: mkdir vs mkdir ======================================================================== 21:21:57 (1746148917) mkdir: cannot create directory '/mnt/lustre2/f42a.sanityn': File exists Conflict PASS 42a (9s) == sanityn test 42b: pdirops: mkdir vs create ======================================================================== 21:22:06 (1746148926) open: File exists Conflict PASS 42b (10s) == sanityn test 42c: pdirops: mkdir vs link ======================================================================== 21:22:17 (1746148937) link: cannot create link '/mnt/lustre2/f42c.sanityn' to '/mnt/lustre2/f42c.sanityn-2': File exists Conflict PASS 42c (11s) == sanityn test 42d: pdirops: mkdir vs unlink ======================================================================== 21:22:27 (1746148947) Conflict PASS 42d (10s) == sanityn test 42e: pdirops: mkdir and rename (tgt) ======================================================================== 21:22:37 (1746148957) mv: cannot overwrite directory '/mnt/lustre2/f42e.sanityn' with non-directory Conflict PASS 42e (9s) == sanityn test 42f: pdirops: mkdir and rename (src) ======================================================================== 21:22:46 (1746148966) Conflict PASS 42f (10s) == sanityn test 42g: pdirops: mkdir vs getattr ======================================================================== 21:22:56 (1746148976) Conflict PASS 42g (10s) == sanityn test 42h: pdirops: mkdir vs readdir ======================================================================== 21:23:06 (1746148986) Conflict PASS 42h (12s) == sanityn test 43a: rmdir,mkdir doesn't return -EEXIST ======================================================================== 21:23:18 (1746148998) PASS 43a (59s) == sanityn test 43b: pdirops: unlink vs create ======================================================================== 21:24:17 (1746149057) Conflict PASS 43b (10s) == sanityn test 43c: pdirops: unlink vs link ======================================================================== 21:24:27 (1746149067) Conflict PASS 43c (9s) == sanityn test 43d: pdirops: unlink vs unlink ======================================================================== 21:24:37 (1746149077) rm: cannot remove '/mnt/lustre2/f43d.sanityn': No such file or directory Conflict PASS 43d (9s) == sanityn test 43e: pdirops: unlink and rename (tgt) ======================================================================== 21:24:46 (1746149086) Conflict PASS 43e (9s) == sanityn test 43f: pdirops: unlink and rename (src) ======================================================================== 21:24:55 (1746149095) mv: cannot stat '/mnt/lustre2/f43f.sanityn': No such file or directory Conflict PASS 43f (9s) == sanityn test 43g: pdirops: unlink vs getattr ======================================================================== 21:25:04 (1746149104) stat: cannot stat '/mnt/lustre2/f43g.sanityn': No such file or directory Conflict PASS 43g (10s) == sanityn test 43h: pdirops: unlink vs readdir ======================================================================== 21:25:14 (1746149114) Conflict PASS 43h (10s) == sanityn test 43i: pdirops: unlink vs remote mkdir ===== 21:25:24 (1746149124) SKIP: sanityn test_43i needs >= 2 MDTs SKIP 43i (2s) == sanityn test 43j: racy mkdir return EEXIST ======================================================================== 21:25:26 (1746149126) mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre2/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists mkdir: cannot create directory '/mnt/lustre/d43j.sanityn/sub': File exists PASS 43j (95s) == sanityn test 43k: unlink vs create ==================== 21:27:01 (1746149221) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 43k (954s) == sanityn test 44a: pdirops: rename tgt vs mkdir ======================================================================== 21:42:55 (1746150175) mkdir: cannot create directory '/mnt/lustre2/f44a.sanityn': File exists Conflict Thu May 1 21:43:02 EDT 2025 PASS 44a (11s) == sanityn test 44b: pdirops: rename tgt vs create ======================================================================== 21:43:07 (1746150187) open: File exists Conflict PASS 44b (12s) == sanityn test 44c: pdirops: rename tgt vs link ======================================================================== 21:43:18 (1746150198) link: cannot create link '/mnt/lustre2/f44c.sanityn' to '/mnt/lustre2/f44c.sanityn-3': File exists Conflict PASS 44c (11s) == sanityn test 44d: pdirops: rename tgt vs unlink ======================================================================== 21:43:30 (1746150210) Conflict PASS 44d (10s) == sanityn test 44e: pdirops: rename tgt and rename (tgt) ======================================================================== 21:43:39 (1746150219) Conflict PASS 44e (10s) == sanityn test 44f: pdirops: rename tgt and rename (src) ======================================================================== 21:43:49 (1746150229) Conflict PASS 44f (10s) == sanityn test 44g: pdirops: rename tgt vs getattr ======================================================================== 21:43:59 (1746150239) Conflict PASS 44g (10s) == sanityn test 44h: pdirops: rename tgt vs readdir ======================================================================== 21:44:10 (1746150250) Conflict PASS 44h (11s) == sanityn test 44i: pdirops: rename tgt vs remote mkdir ========================================================== 21:44:20 (1746150260) SKIP: sanityn test_44i needs >= 2 MDTs SKIP 44i (3s) == sanityn test 45a: rename,mkdir doesn't return -EEXIST ======================================================================== 21:44:23 (1746150263) PASS 45a (110s) == sanityn test 45b: pdirops: rename src vs create ======================================================================== 21:46:13 (1746150373) Conflict PASS 45b (9s) == sanityn test 45c: pdirops: rename src vs link ======================================================================== 21:46:22 (1746150382) Conflict PASS 45c (9s) == sanityn test 45d: pdirops: rename src vs unlink ======================================================================== 21:46:31 (1746150391) rm: cannot remove '/mnt/lustre2/f45d.sanityn': No such file or directory Conflict PASS 45d (10s) == sanityn test 45e: pdirops: rename src and rename (tgt) ======================================================================== 21:46:41 (1746150401) Conflict PASS 45e (10s) == sanityn test 45f: pdirops: rename src and rename (src) ======================================================================== 21:46:51 (1746150411) mv: cannot stat '/mnt/lustre2/f45f.sanityn': No such file or directory Conflict PASS 45f (10s) == sanityn test 45g: pdirops: rename src vs getattr ======================================================================== 21:47:01 (1746150421) stat: cannot stat '/mnt/lustre2/f45g.sanityn': No such file or directory Conflict PASS 45g (9s) == sanityn test 45h: pdirops: unlink vs readdir ======================================================================== 21:47:10 (1746150430) Conflict PASS 45h (9s) == sanityn test 45i: pdirops: rename src vs remote mkdir ========================================================== 21:47:19 (1746150439) SKIP: sanityn test_45i needs >= 2 MDTs SKIP 45i (3s) == sanityn test 45j: read vs rename ====================== 21:47:22 (1746150442) Begin 100 tests with fail_loc=0x80000169 Progress: **************************************************************************************************** Begin 100 tests with fail_loc=0x8000016a Progress: **************************************************************************************************** PASS 45j (860s) == sanityn test 46a: pdirops: link vs mkdir ======================================================================== 22:01:42 (1746151302) mkdir: cannot create directory '/mnt/lustre2/f46a.sanityn': File exists Conflict PASS 46a (10s) == sanityn test 46b: pdirops: link vs create ======================================================================== 22:01:52 (1746151312) open: File exists Conflict PASS 46b (9s) == sanityn test 46c: pdirops: link vs link ======================================================================== 22:02:01 (1746151321) link: cannot create link '/mnt/lustre2/f46c.sanityn' to '/mnt/lustre2/f46c.sanityn': File exists Conflict PASS 46c (9s) == sanityn test 46d: pdirops: link vs unlink ======================================================================== 22:02:10 (1746151330) Conflict PASS 46d (10s) == sanityn test 46e: pdirops: link and rename (tgt) ======================================================================== 22:02:20 (1746151340) Conflict PASS 46e (10s) == sanityn test 46f: pdirops: link and rename (src) ======================================================================== 22:02:31 (1746151351) Conflict PASS 46f (11s) == sanityn test 46g: pdirops: link vs getattr ======================================================================== 22:02:41 (1746151361) Conflict PASS 46g (10s) == sanityn test 46h: pdirops: link vs readdir ======================================================================== 22:02:51 (1746151371) Conflict PASS 46h (11s) == sanityn test 46i: pdirops: link vs remote mkdir ======= 22:03:02 (1746151382) SKIP: sanityn test_46i needs >= 2 MDTs SKIP 46i (3s) == sanityn test 47a: pdirops: remote mkdir vs mkdir ====== 22:03:05 (1746151385) SKIP: sanityn test_47a needs >= 2 MDTs SKIP 47a (3s) == sanityn test 47b: pdirops: remote mkdir vs create ===== 22:03:08 (1746151388) SKIP: sanityn test_47b needs >= 2 MDTs SKIP 47b (2s) == sanityn test 47c: pdirops: remote mkdir vs link ======= 22:03:10 (1746151390) SKIP: sanityn test_47c needs >= 2 MDTs SKIP 47c (2s) == sanityn test 47d: pdirops: remote mkdir vs unlink ===== 22:03:13 (1746151393) SKIP: sanityn test_47d needs >= 2 MDTs SKIP 47d (3s) == sanityn test 47e: pdirops: remote mkdir and rename (tgt) ========================================================== 22:03:15 (1746151395) SKIP: sanityn test_47e needs >= 2 MDTs SKIP 47e (2s) == sanityn test 47f: pdirops: remote mkdir and rename (src) ========================================================== 22:03:18 (1746151398) SKIP: sanityn test_47f needs >= 2 MDTs SKIP 47f (2s) == sanityn test 47g: pdirops: remote mkdir vs getattr ==== 22:03:21 (1746151401) SKIP: sanityn test_47g needs >= 2 MDTs SKIP 47g (3s) == sanityn test 50: osc lvb attrs: enqueue vs. CP AST ======================================================================== 22:03:23 (1746151403) 10+0 records in 10+0 records out 10240 bytes (10 kB) copied, 0.176938 s, 57.9 kB/s fail_loc=0x410 fail_loc=0x0 PASS 50 (10s) == sanityn test 51a: layout lock: refresh layout should work ========================================================== 22:03:33 (1746151413) 0+1 records in 0+1 records out 159 bytes (159 B) copied, 0.0215154 s, 7.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 3519: kill: (11644) - No such process PASS 51a (7s) == sanityn test 51b: layout lock: glimpse should be able to restart if layout changed ========================================================== 22:03:40 (1746151420) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0102934 s, 99.5 kB/s fail_loc=0x1404 fail_val=4 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0757944 s, 13.8 MB/s 3145728 PASS 51b (17s) == sanityn test 51c: layout lock: IT_LAYOUT blocked and correct layout can be returned ========================================================== 22:03:57 (1746151437) fail_loc=0x172 Setting layout to have 2 stripes ... 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0255411 s, 40.1 kB/s PASS 51c (9s) == sanityn test 51d: layout lock: losing layout lock should clean up memory map region ========================================================== 22:04:06 (1746151446) 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.027881 s, 37.6 MB/s Before revoking layout lock: 1024 KB mapped PASS 51d (6s) == sanityn test 51e: lfs getstripe does not break leases, part 2 ========================================================== 22:04:13 (1746151453) /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2443 0x98b 0x240000400 write lease(2) has applied. /mnt/lustre2/f51e.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 2443 0x98b 0x240000400 read lease(1) has applied. PASS 51e (6s) == sanityn test 54: rename locking ======================= 22:04:20 (1746151460) llite.lustre-ffff8800aa215800.xattr_cache=1 llite.lustre-ffff8800b6e70800.xattr_cache=1 ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505005 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-01 22:04:21.000000000 -0400 Modify: 2025-05-01 22:04:21.000000000 -0400 Change: 2025-05-01 22:04:21.000000000 -0400 Birth: - ==> rename vs getattr vs setxattr should not deadlock fail_loc=0x80000154 File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505008 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-01 22:04:27.000000000 -0400 Modify: 2025-05-01 22:04:27.000000000 -0400 Change: 2025-05-01 22:04:27.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000153 File: '/mnt/lustre/d1/d2' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505011 Links: 3 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-01 22:04:33.000000000 -0400 Modify: 2025-05-01 22:04:33.000000000 -0400 Change: 2025-05-01 22:04:33.000000000 -0400 Birth: - open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505010 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-01 22:04:33.000000000 -0400 Modify: 2025-05-01 22:04:33.000000000 -0400 Change: 2025-05-01 22:04:33.000000000 -0400 Birth: - ==> rename vs getattr vs open vs getattr should not deadlock fail_loc=0x80000154 open(O_RDWR|O_CREAT): Is a directory File: '/mnt/lustre/d1/d2' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505014 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-01 22:04:39.000000000 -0400 Modify: 2025-05-01 22:04:40.000000000 -0400 Change: 2025-05-01 22:04:40.000000000 -0400 Birth: - File: '/mnt/lustre/d1' Size: 11776 Blocks: 23 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505013 Links: 4 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-01 22:04:39.000000000 -0400 Modify: 2025-05-01 22:04:40.000000000 -0400 Change: 2025-05-01 22:04:40.000000000 -0400 Birth: - PASS 54 (29s) == sanityn test 55a: rename vs unlink target dir ========= 22:04:49 (1746151489) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55a.sanityn/d1/d2' to '/mnt/lustre/d55a.sanityn/d3/d2': No such file or directory PASS 55a (11s) == sanityn test 55b: rename vs unlink source dir ========= 22:05:00 (1746151500) fail_loc=0x80000156 mv: cannot move '/mnt/lustre/d55b.sanityn/d1/d2' to '/mnt/lustre/d55b.sanityn/d3/d2': No such file or directory PASS 55b (11s) == sanityn test 55c: rename vs unlink orphan target dir == 22:05:11 (1746151511) fail_loc=0x156 mv: cannot move '/mnt/lustre/d55c.sanityn/d1/d2' to '/mnt/lustre/d55c.sanityn/d3/d2': No such file or directory PASS 55c (15s) == sanityn test 55d: rename file vs link ================= 22:05:26 (1746151526) fail_loc=0x155 mv: cannot move '/mnt/lustre/d55d.sanityn/f1' to '/mnt/lustre/d55d.sanityn/d55d.sanityn': Is a directory PASS 55d (13s) == sanityn test 56a: test llverdev with single large stripe ========================================================== 22:05:39 (1746151539) run_llverdev /mnt/lustre/f56a.sanityn -p -s 16000000 -c 4k llverdev: /mnt/lustre/f56a.sanityn is 16777216000000 bytes (15625 GB) in size timestamp: 1746151541 chunksize: 4096 size: 16777216000000 write offset: 16384000000kB 0.803146 MB/s read offset: 16384000000kB inf MB/s llverdev: data verified successfully PASS 56a (82s) == sanityn test 56b: test llverdev and partial verify of wide stripe file ========================================================== 22:07:01 (1746151621) run_llverdev /mnt/lustre/f56b.sanityn -p -s 32000000 -c 4k llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: read /mnt/lustre/f56b.sanityn@33554431995904+4096 no data: Success llverdev: /mnt/lustre/f56b.sanityn is 33554432000000 bytes (31250 GB) in size timestamp: 1746151624 chunksize: 4096 size: 33554432000000 write offset: 32768000000kB 1.67225 MB/s read offset: 32768000000kB -nan MB/s llverdev: data verified successfully PASS 56b (82s) == sanityn test 60: Verify data_version behaviour ======== 22:08:23 (1746151703) PASS 60 (4s) == sanityn test 70a: cd directory && rm directory ======== 22:08:27 (1746151707) PASS 70a (4s) == sanityn test 70b: remove files after calling rm_entry ========================================================== 22:08:31 (1746151711) PASS 70b (5s) == sanityn test 71a: correct file map just after write operation is finished ========================================================== 22:08:36 (1746151716) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00920137 s, 7.1 MB/s SKIP: sanityn test_71a ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71a (2s) == sanityn test 71b: check fiemap support for stripecount > 1 ========================================================== 22:08:38 (1746151718) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.00861358 s, 7.6 MB/s SKIP: sanityn test_71b ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71b (2s) == sanityn test 71c: check FIEMAP_EXTENT_LAST flag with different extents number ========================================================== 22:08:41 (1746151721) SKIP: sanityn test_71c support only ldiskfs ost SKIP 71c (1s) == sanityn test 71d: fiemap corruption test with fm_extent_count=0 ========================================================== 22:08:42 (1746151722) 1+0 records in 1+0 records out 65536 bytes (66 kB) copied, 0.0122637 s, 5.3 MB/s SKIP: sanityn test_71d ORI-366/LU-1941: FIEMAP unimplemented on ZFS SKIP 71d (2s) == sanityn test 72: getxattr/setxattr cache should be consistent between nodes ========================================================== 22:08:44 (1746151724) llite.lustre-ffff8800aa215800.xattr_cache=1 llite.lustre-ffff8800b6e70800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names user.attr1="value1" getfattr: Removing leading '/' from absolute path names user.attr1="value2" getfattr: Removing leading '/' from absolute path names getfattr: Removing leading '/' from absolute path names b362740ad371eae580e7f85452e8830b - 870e072da1ebfc20ea8c0cd3d6bdf3b6 - PASS 72 (4s) == sanityn test 73: getxattr should not cause xattr lock cancellation ========================================================== 22:08:48 (1746151728) llite.lustre-ffff8800aa215800.xattr_cache=1 llite.lustre-ffff8800b6e70800.xattr_cache=1 getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/f73.sanityn user.attr1="value1" getfattr: Removing leading '/' from absolute path names # file: mnt/lustre2/f73.sanityn user.attr1="value1" PASS 73 (5s) == sanityn test 74: flock deadlock: different mounts ======================================================================== 22:08:53 (1746151733) 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.00732474 s, 140 kB/s 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.0128133 s, 79.9 kB/s 24197: get lock2 24198: get lock1 24197: done 24197: get lock1 24198: done 24198: get lock2 fcntl cmd 7 failed: Resource deadlock avoided 24198: cannot get lock2: Resource deadlock avoided 24198: exit rc=0 24197: done 24197: exit rc=0 PASS 74 (9s) == sanityn test 75: osc: upcall after unuse lock============================================================================= 22:09:02 (1746151742) 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.0388321 s, 54.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413903 s, 25.3 MB/s fail_loc=0x31d /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 4108: 24840 Killed stat -c %s $DIR1/$tfile fail_loc=0 PASS 75 (10s) == sanityn test 76: Verify MDT open_files listing ======== 22:09:12 (1746151752) opening files: ........ closing files: ........ PASS 76 (47s) == sanityn test 77a: check FIFO NRS policy =============== 22:09:59 (1746151799) ost.OSS.ost.nrs_policies=fifo ost.OSS.ost_create.nrs_policies=fifo ost.OSS.ost_io.nrs_policies=fifo ost.OSS.ost_out.nrs_policies=fifo ost.OSS.ost_seq.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.180407 s, 93.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0282147 s, 37.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241954 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.044665 s, 23.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0732523 s, 14.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0985533 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0958042 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.111552 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106008 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.115078 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121351 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.117844 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.103788 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.132975 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.10649 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.139082 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.128945 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242126 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0057979 s, 181 MB/s , 0.0453055 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0504193 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00883689 s, 119 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00902318 s, 116 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0107094 s, 97.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0454234 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.039586 s, 26.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00195129 s, 537 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00249468 s, 420 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00585669 s, 179 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00728231 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0184065 s, 57.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0408411 s, 25.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0512318 s, 20.5 MB/s removed '/mnt/lustre/d77a.sanityn/nrs_r_oleg120-client.virtnet' removed '/mnt/lustre/d77a.sanityn/nrs_w_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77a.sanityn' PASS 77a (6s) == sanityn test 77b: check CRR-N NRS policy ============== 22:10:05 (1746151805) ost.OSS.ost.nrs_policies=crrn ost.OSS.ost_create.nrs_policies=crrn ost.OSS.ost_io.nrs_policies=crrn ost.OSS.ost_out.nrs_policies=crrn ost.OSS.ost_seq.nrs_policies=crrn ost.OSS.ost.nrs_crrn_quantum=1 ost.OSS.ost_create.nrs_crrn_quantum=1 ost.OSS.ost_io.nrs_crrn_quantum=1 ost.OSS.ost_out.nrs_crrn_quantum=1 ost.OSS.ost_seq.nrs_crrn_quantum=1 policy: crr-n, crrn_quantum 1 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.223045 s, 75.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0255422 s, 41.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0314457 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0919061 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0956158 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0639956 s, 16.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.110114 s, 9.5 MB/s 1048576 bytes (1.0 MB) copied, 0.108694 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0901893 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.111873 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.115773 s, 9.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1294 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.114895 s, 9.1 MB/s 1048576 bytes (1.0 MB) copied, 0.135553 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.134424 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.126971 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.125184 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0392938 s, 26.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00232658 s, 451 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0230035 s, 45.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0202258 s, 51.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0301345 s, 34.8 MB/s , 0.00419613 s, 250 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0152162 s, 68.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0666217 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0153108 s, 68.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0394713 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0471875 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0475573 s, 22.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.048173 s, 21.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0150393 s, 69.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0536227 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00675704 s, 155 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg120-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost.nrs_crrn_quantum=64 ost.OSS.ost_create.nrs_crrn_quantum=64 ost.OSS.ost_io.nrs_crrn_quantum=64 ost.OSS.ost_out.nrs_crrn_quantum=64 ost.OSS.ost_seq.nrs_crrn_quantum=64 policy: crr-n, crrn_quantum 64 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.236794 s, 70.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356891 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0662192 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.117712 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.129363 s, 8.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157013 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152522 s, 6.9 MB/s 1048576 bytes (1.0 MB) copied, 0.152619 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.162413 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.159001 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166109 s, 6.3 MB/s , 0.155685 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.170858 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.171038 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.182617 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.177586 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.194528 s, 5.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00205251 s, 511 MB/s , 0.0192471 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00935275 s, 112 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00643949 s, 163 MB/s 1048576 bytes (1.0 MB) copied, 0.00533748 s, 196 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00194005 s, 540 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00435849 s, 241 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00179487 s, 584 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0954968 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0858571 s, 12.2 MB/s , 0.0698534 s, 15.0 MB/s 1048576 bytes (1.0 MB) copied, 0.00607229 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0642281 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00419377 s, 250 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00563927 s, 186 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202363 s, 51.8 MB/s removed '/mnt/lustre/d77b.sanityn/nrs_w_oleg120-client.virtnet' removed '/mnt/lustre/d77b.sanityn/nrs_r_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77b.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77b (8s) == sanityn test 77c: check ORR NRS policy ================ 22:10:13 (1746151813) ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_orr_quantum=1 ost.OSS.ost_io.nrs_orr_offset_type=physical ost.OSS.ost_io.nrs_orr_supported=reads policy: orr, orr_quantum 1, orr_offset_type physical, orr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.208809 s, 80.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231018 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259836 s, 40.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0268496 s, 39.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0372653 s, 28.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0915701 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.117843 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.103551 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0968315 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.125682 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150491 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.138877 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.139448 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146675 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143863 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166296 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.163256 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0164684 s, 63.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0369786 s, 28.4 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b6e70800.lock_unused_count=1 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127455 s, 82.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.022114 s, 47.4 MB/s , 0.00403371 s, 260 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00190121 s, 552 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0262763 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280749 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0303354 s, 34.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0326286 s, 32.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331264 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00825338 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0298802 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.038992 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0430717 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0047256 s, 222 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg120-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=writes ost.OSS.ost_io.nrs_orr_quantum=64 policy: orr, orr_quantum 64, orr_offset_type physical, orr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.209472 s, 80.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0313327 s, 33.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0300505 s, 34.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.101284 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.100343 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.103042 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.104214 s, 10.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.123827 s, 8.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.127459 s, 8.2 MB/s , 0.12138 s, 8.6 MB/s 1048576 bytes (1.0 MB) copied, 0.113684 s, 9.2 MB/s 1048576 bytes (1.0 MB) copied, 0.130833 s, 8.0 MB/s 1048576 bytes (1.0 MB) copied, 0.1182 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.126293 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.12172 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.123697 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.141695 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273073 s, 38.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.040188 s, 26.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0272463 s, 38.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0323792 s, 32.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316116 s, 33.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0321139 s, 32.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0559809 s, 18.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0511313 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0346216 s, 30.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0384117 s, 27.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0375796 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0394846 s, 26.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0410646 s, 25.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0442299 s, 23.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0512271 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0729287 s, 14.4 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg120-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_orr_supported=reads_and_writes ost.OSS.ost_io.nrs_orr_offset_type=logical policy: orr, orr_quantum 64, orr_offset_type logical, orr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.221352 s, 75.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220013 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464959 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0541651 s, 19.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0570607 s, 18.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143503 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.168716 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.136322 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.131755 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153327 s, 6.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.230918 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188317 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.241985 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.252607 s, 4.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.240082 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.244903 s, 4.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.267422 s, 3.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00362874 s, 289 MB/s , 0.00357638 s, 293 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00201109 s, 521 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0636514 s, 16.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0703981 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0848442 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00313282 s, 335 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0544944 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0643936 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0376525 s, 27.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0867209 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0492225 s, 21.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0360805 s, 29.1 MB/s 1048576 bytes (1.0 MB) copied, 0.0893088 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0347328 s, 30.2 MB/s , 0.0170656 s, 61.4 MB/s removed '/mnt/lustre/d77c.sanityn/nrs_w_oleg120-client.virtnet' removed '/mnt/lustre/d77c.sanityn/nrs_r_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77c.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77c (9s) == sanityn test 77d: check TRR nrs policy ================ 22:10:22 (1746151822) ost.OSS.ost_io.nrs_policies=trr ost.OSS.ost_io.nrs_trr_quantum=1 ost.OSS.ost_io.nrs_trr_offset_type=physical ost.OSS.ost_io.nrs_trr_supported=reads policy: trr, trr_quantum 1, trr_offset_type physical, trr_supported reads 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.241988 s, 69.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0270149 s, 38.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0473064 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0739889 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0922641 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0988108 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.108917 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.117268 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121178 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.117476 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.125086 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.123964 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.128289 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.134498 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.144835 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143747 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.172628 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0337261 s, 31.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00227129 s, 462 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00607189 s, 173 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00621562 s, 169 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.015994 s, 65.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219979 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0192265 s, 54.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00571736 s, 183 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162734 s, 64.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241934 s, 43.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00438003 s, 239 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0517921 s, 20.2 MB/s , 0.0309285 s, 33.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0545349 s, 19.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0405353 s, 25.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00779785 s, 134 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg120-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=writes ost.OSS.ost_io.nrs_trr_quantum=64 policy: trr, trr_quantum 64, trr_offset_type physical, trr_supported writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.232949 s, 72.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.028131 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0345228 s, 30.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0389787 s, 26.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0354379 s, 29.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145531 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156827 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146352 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166112 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.175003 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.19638 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.177126 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.170665 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.179813 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.18991 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.21594 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.225063 s, 4.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319216 s, 32.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0349347 s, 30.0 MB/s 1048576 bytes (1.0 MB) copied, 0.0231062 s, 45.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240511 s, 43.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0128978 s, 81.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0156108 s, 67.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0159623 s, 65.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0174503 s, 60.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175243 s, 59.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0186159 s, 56.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0215634 s, 48.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.021119 s, 49.7 MB/s 1048576 bytes (1.0 MB) copied, 0.021931 s, 47.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219634 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0220055 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273198 s, 38.4 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg120-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_trr_supported=reads_and_writes ost.OSS.ost_io.nrs_trr_offset_type=logical policy: trr, trr_quantum 64, trr_offset_type logical, trr_supported reads_and_writes 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.219893 s, 76.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018679 s, 56.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0258407 s, 40.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0688108 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0803051 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0692666 s, 15.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0838394 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0857389 s, 12.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0920227 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0914816 s, 11.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.113813 s, 9.2 MB/s 1048576 bytes (1.0 MB) copied, 0.103443 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.126111 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.113988 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.120703 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1327 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149622 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278405 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00374043 s, 280 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364115 s, 28.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0367805 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0435463 s, 24.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0587747 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00257684 s, 407 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00380295 s, 276 MB/s , 0.00552387 s, 190 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0632421 s, 16.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0480993 s, 21.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0486218 s, 21.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0427504 s, 24.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0203475 s, 51.5 MB/s , 0.0149231 s, 70.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0633978 s, 16.5 MB/s removed '/mnt/lustre/d77d.sanityn/nrs_r_oleg120-client.virtnet' removed '/mnt/lustre/d77d.sanityn/nrs_w_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77d.sanityn' ost.OSS.ost_io.nrs_policies=fifo PASS 77d (9s) == sanityn test 77e: check TBF NID nrs policy ============ 22:10:31 (1746151831) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_tbf_rule=start localhost nid={0@lo} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start clients nid={192.168.201.20@tcp} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start others nid={*.*.*.*@tcp} rate=50 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.230198 s, 72.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365442 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464113 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.127885 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.132935 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.138077 s, 7.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14328 s, 7.3 MB/s 1048576 bytes (1.0 MB) copied, 0.138422 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.140426 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.140736 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143816 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146725 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.144269 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142304 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142041 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.149141 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166301 s, 6.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00452853 s, 232 MB/s , 0.0399414 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109099 s, 96.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00493639 s, 212 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266541 s, 39.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00353912 s, 296 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00406701 s, 258 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00450732 s, 233 MB/s 1048576 bytes (1.0 MB) copied, 0.00451326 s, 232 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0246928 s, 42.5 MB/s 1048576 bytes (1.0 MB) copied, 0.0182685 s, 57.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0332663 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00801681 s, 131 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00967408 s, 108 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00968458 s, 108 MB/s , 0.00389839 s, 269 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg120-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change localhost rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change clients rate=101 ost.OSS.ost_io.nrs_tbf_rule=change others rate=51 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.201042 s, 83.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0102793 s, 102 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0457131 s, 22.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011165 s, 93.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0555019 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.180041 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.183045 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.186256 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.179142 s, 5.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.206661 s, 5.1 MB/s 1048576 bytes (1.0 MB) copied, 0.1995 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.207597 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.380378 s, 2.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.381604 s, 2.7 MB/s 1048576 bytes (1.0 MB) copied, 0.375239 s, 2.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.381969 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.392644 s, 2.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0383907 s, 27.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00328644 s, 319 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.016698 s, 62.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0162957 s, 64.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0038793 s, 270 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138752 s, 75.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0262931 s, 39.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219965 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0462363 s, 22.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0288604 s, 36.3 MB/s 1048576 bytes (1.0 MB) copied, 0.029169 s, 35.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0298406 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249889 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00508111 s, 206 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00917272 s, 114 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0123737 s, 84.7 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg120-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop localhost ost.OSS.ost_io.nrs_tbf_rule=stop clients ost.OSS.ost_io.nrs_tbf_rule=stop others 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.149496 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0104318 s, 101 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0239435 s, 43.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276115 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0314584 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335721 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208676 s, 50.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0273689 s, 38.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0171768 s, 61.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0349541 s, 30.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0237438 s, 44.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.018104 s, 57.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.125303 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.11492 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.131692 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.115607 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.169173 s, 6.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0329605 s, 31.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0235788 s, 44.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242476 s, 43.2 MB/s 1048576 bytes (1.0 MB) copied, 0.0236152 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.02562 s, 40.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0284696 s, 36.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390991 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317239 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.032824 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out , 0.0330503 s, 31.7 MB/s 1048576 bytes (1.0 MB) copied, 0.0309905 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.029564 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0266001 s, 39.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0305716 s, 34.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0259883 s, 40.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0250226 s, 41.9 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg120-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' ost.OSS.ost_io.nrs_policies=fifo 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.178247 s, 94.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212289 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0324221 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0652012 s, 16.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0612603 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0686311 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0791766 s, 13.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0824712 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0885307 s, 11.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.092478 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0810511 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0721457 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0983272 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105847 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0913347 s, 11.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0904289 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.111831 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00401854 s, 261 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0287883 s, 36.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0243164 s, 43.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0154643 s, 67.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00697311 s, 150 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.020541 s, 51.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00823819 s, 127 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00722632 s, 145 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0242115 s, 43.3 MB/s , 0.00884966 s, 118 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140902 s, 74.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0125776 s, 83.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00958414 s, 109 MB/s , 0.00776312 s, 135 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00567381 s, 185 MB/s 1048576 bytes (1.0 MB) copied, 0.00229554 s, 457 MB/s removed '/mnt/lustre/d77e.sanityn/nrs_r_oleg120-client.virtnet' removed '/mnt/lustre/d77e.sanityn/nrs_w_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77e.sanityn' PASS 77e (13s) == sanityn test 77f: check TBF JobID nrs policy ========== 22:10:44 (1746151844) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start runas jobid={iozone.500 dd.500 tiotest.500} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start iozone_runas jobid={iozone.500} rate=100 ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.259491 s, 64.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253414 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0390014 s, 26.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0685879 s, 15.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0662521 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.079769 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0949305 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0938985 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.097719 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0813136 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.107726 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0901219 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.100283 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.117766 s, 8.9 MB/s , 0.0803757 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.108007 s, 9.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.120835 s, 8.7 MB/s ldlm.namespaces.lustre-OST0000-osc-ffff8800b6e70800.lock_unused_count=5 ldlm.namespaces.lustre-OST0001-osc-ffff8800b6e70800.lock_unused_count=1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.221487 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.216974 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.207293 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.21863 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.224888 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.210864 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.229995 s, 4.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.222404 s, 4.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.218975 s, 4.8 MB/s 1048576 bytes (1.0 MB) copied, 0.218774 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.220939 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.222019 s, 4.7 MB/s 1048576 bytes (1.0 MB) copied, 0.221682 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.221302 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.223792 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.226266 s, 4.6 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg120-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=change runas rate=1001 ost.OSS.ost_io.nrs_tbf_rule=change iozone_runas rate=101 ost.OSS.ost_io.nrs_tbf_rule=change dd_runas rate=51 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.168334 s, 99.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0229454 s, 45.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0276336 s, 37.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213266 s, 49.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0159286 s, 65.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0208106 s, 50.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0295114 s, 35.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.144547 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.146138 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.18128 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.188853 s, 5.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.181072 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.173098 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.201071 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.201993 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.213155 s, 4.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.259825 s, 4.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0280782 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0201354 s, 52.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0202542 s, 51.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0248824 s, 42.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0261399 s, 40.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0271759 s, 38.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0290303 s, 36.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.031829 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0335681 s, 31.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.035842 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.035825 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.032485 s, 32.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0358355 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0356876 s, 29.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0417016 s, 25.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0522847 s, 20.1 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg120-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_tbf_rule=stop runas ost.OSS.ost_io.nrs_tbf_rule=stop iozone_runas ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.135937 s, 123 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0217911 s, 48.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307379 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0648094 s, 16.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0743945 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0729801 s, 14.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0548747 s, 19.1 MB/s , 0.0478162 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.072785 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0824329 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0842826 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0873708 s, 12.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0819752 s, 12.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0742639 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0836657 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0804267 s, 13.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0921431 s, 11.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out , 0.0226498 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00265046 s, 396 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0223414 s, 46.9 MB/s 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0109452 s, 95.8 MB/s , 0.0508312 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0524203 s, 20.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0294204 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00932381 s, 112 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0138275 s, 75.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0241427 s, 43.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0015537 s, 675 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00189815 s, 552 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00234166 s, 448 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00458233 s, 229 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0705928 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00196572 s, 533 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg120-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' ost.OSS.ost_io.nrs_policies=fifo running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77f.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.187658 s, 89.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.025231 s, 41.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265227 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352069 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0467315 s, 22.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0618471 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0587758 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0614255 s, 17.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0716752 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0742987 s, 14.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0958407 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0958333 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0843044 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0993651 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.083917 s, 12.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0933228 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.102253 s, 10.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0279056 s, 37.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0158656 s, 66.1 MB/s , 0.0151805 s, 69.1 MB/s , 0.0156211 s, 67.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0240053 s, 43.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0176482 s, 59.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0187791 s, 55.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0188689 s, 55.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0211232 s, 49.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0223047 s, 47.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.023282 s, 45.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0227426 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.024878 s, 42.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0231624 s, 45.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238922 s, 43.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0319077 s, 32.9 MB/s removed '/mnt/lustre/d77f.sanityn/nrs_r_oleg120-client.virtnet' removed '/mnt/lustre/d77f.sanityn/nrs_w_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77f.sanityn' PASS 77f (12s) == sanityn test 77g: Change TBF type directly ============ 22:10:56 (1746151856) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.500} rate=50 ost.OSS.ost_io.nrs_policies=fifo PASS 77g (6s) == sanityn test 77h: Wrong policy name should report error, not LBUG ========================================================== 22:11:02 (1746151862) oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=abc: No such file or directory oleg120-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='abc': No such file or directory pdsh@oleg120-client: oleg120-server: ssh exited with exit code 2 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc: Unknown error 524 oleg120-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc': Unknown error 524 pdsh@oleg120-client: oleg120-server: ssh exited with exit code 12 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf reg abc: Unknown error 524 oleg120-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf reg abc': Unknown error 524 pdsh@oleg120-client: oleg120-server: ssh exited with exit code 12 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_policies=tbf abc efg: Unknown error 524 oleg120-server: error: set_param: setting 'ost/OSS/ost_io/nrs_policies'='tbf abc efg': Unknown error 524 pdsh@oleg120-client: oleg120-server: ssh exited with exit code 12 PASS 77h (7s) == sanityn test 77i: Change rank of TBF rule ============= 22:11:09 (1746151869) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=start before jobid={jobid} rate=1000 ost.OSS.ost_io.nrs_tbf_rule=start after jobid={jobid} rate=1000 rank=default ost.OSS.ost_io.nrs_tbf_rule=start target jobid={jobid} rate=1000 rank=after Move before itself ost.OSS.ost_io.nrs_tbf_rule=change target rank=target Move to higher rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=before Move to lower rank ost.OSS.ost_io.nrs_tbf_rule=change target rank=after Move before default ost.OSS.ost_io.nrs_tbf_rule=change target rank=default ost.OSS.ost_io.nrs_policies=fifo PASS 77i (13s) == sanityn test 77j: check TBF-OPCode NRS policy ========= 22:11:23 (1746151883) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf opcode ost.OSS.ost_io.nrs_tbf_rule=start ost_r opcode={ost_read} rate=5 ost.OSS.ost_io.nrs_tbf_rule=start ost_w opcode={ost_write} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.391387 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0236197 s, 44.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.031643 s, 33.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0863124 s, 12.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0826792 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0946592 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0883549 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0824497 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.104734 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.109076 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1226 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.109249 s, 9.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.115677 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.150204 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157743 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.192015 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.214962 s, 4.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0394359 s, 26.6 MB/s 1048576 bytes (1.0 MB) copied, 0.00457701 s, 229 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0366888 s, 28.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0079479 s, 132 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203851 s, 51.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0212232 s, 49.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0335116 s, 31.3 MB/s , 0.0245711 s, 42.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0249408 s, 42.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0044658 s, 235 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0238275 s, 44.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0331181 s, 31.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0241054 s, 43.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0244665 s, 42.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0265418 s, 39.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0303338 s, 34.6 MB/s removed '/mnt/lustre/d77j.sanityn/nrs_w_oleg120-client.virtnet' removed '/mnt/lustre/d77j.sanityn/nrs_r_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77j.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 5 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 6.8434 s, 15.3 MB/s Write runtime is 8 s, speed is 12.500000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.5207 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77j.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_r ost.OSS.ost_io.nrs_tbf_rule=stop ost_w ost.OSS.ost_io.nrs_policies=fifo PASS 77j (58s) == sanityn test 77ja: check TBF-UID/GID NRS policy ======= 22:12:21 (1746151941) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg120-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg120-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg120-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg120-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.27517 s, 13.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.050473 s, 20.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0477507 s, 22.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0885118 s, 11.8 MB/s , 0.0779174 s, 13.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.112116 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.113823 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.124831 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.11884 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.132898 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.156826 s, 6.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.162753 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.158156 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.162278 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.23954 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.240572 s, 4.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.246182 s, 4.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.043228 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.044418 s, 23.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0482629 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0465193 s, 22.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0479455 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0500343 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.058863 s, 17.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0532025 s, 19.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.062429 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0565447 s, 18.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0704465 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0615243 s, 17.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0738201 s, 14.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.065403 s, 16.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0781819 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0750922 s, 14.0 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg120-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8265 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8836 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg120-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg120-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [chmod] [777] [/mnt/lustre/d77ja.sanityn/nrs_r_oleg120-client.virtnet] [/mnt/lustre/d77ja.sanityn/nrs_w_oleg120-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.2478 s, 13.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0429466 s, 24.4 MB/s 1048576 bytes (1.0 MB) copied, 0.0410065 s, 25.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0464292 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0460749 s, 22.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.051556 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495685 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0602057 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0997409 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.110749 s, 9.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.133247 s, 7.9 MB/s 1048576 bytes (1.0 MB) copied, 0.13894 s, 7.5 MB/s 1048576 bytes (1.0 MB) copied, 0.149752 s, 7.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.1629 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.16808 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.164182 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.174487 s, 6.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0213714 s, 49.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0219651 s, 47.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338241 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.024298 s, 43.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0362716 s, 28.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0298945 s, 35.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328256 s, 31.9 MB/s 1048576 bytes (1.0 MB) copied, 0.0336621 s, 31.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0354462 s, 29.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0440286 s, 23.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0442127 s, 23.7 MB/s , 0.0294509 s, 35.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0473372 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0490195 s, 21.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.050978 s, 20.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0536692 s, 19.5 MB/s removed '/mnt/lustre/d77ja.sanityn/nrs_r_oleg120-client.virtnet' removed '/mnt/lustre/d77ja.sanityn/nrs_w_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77ja.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [touch] [/mnt/lustre/d77ja.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77ja.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8555 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77ja.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8718 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77ja.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77ja (135s) == sanityn test 77jb: check TBF-UID/GID NRS policy on files that don't belong to us ========================================================== 22:14:36 (1746152076) jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf uid ost.OSS.ost_io.nrs_tbf_rule=start ost_uid uid={500} rate=5 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg120-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg120-client.virtnet] running as uid/gid/euid/egid 0/500/0/500, groups: 500 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg120-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg120-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.46008 s, 11.5 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0463301 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0496096 s, 21.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0453112 s, 23.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0592957 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.103188 s, 10.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.100597 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.111523 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.124703 s, 8.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.115288 s, 9.1 MB/s 1048576 bytes (1.0 MB) copied, 0.136272 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.160157 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.138856 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.172438 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.173757 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.171261 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.209258 s, 5.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342885 s, 30.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.038109 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.040702 s, 25.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423858 s, 24.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.043215 s, 24.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0448725 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.044744 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464466 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0472895 s, 22.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.049844 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0603756 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0608431 s, 17.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0582231 s, 18.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0625923 s, 16.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0628951 s, 16.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.072197 s, 14.5 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg120-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 0/500/0/500, groups: 500 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8628 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8957 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_uid ost.OSS.ost_io.nrs_policies=fifo jobid_var=procname_uid ost.OSS.ost_io.nrs_policies=tbf gid ost.OSS.ost_io.nrs_tbf_rule=start ost_gid gid={500} rate=5 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg120-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg120-client.virtnet] running as uid/gid/euid/egid 500/0/500/0, groups: 0 [chmod] [777] [/mnt/lustre/d77jb.sanityn/nrs_r_oleg120-client.virtnet] [/mnt/lustre/d77jb.sanityn/nrs_w_oleg120-client.virtnet] running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 1.24988 s, 13.4 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0727616 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0829296 s, 12.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0893661 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.115122 s, 9.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.131684 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.121531 s, 8.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.107031 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.11744 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.120294 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.130572 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.165312 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.163598 s, 6.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.166614 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.181014 s, 5.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.20851 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.20562 s, 5.1 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0278479 s, 37.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0512598 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328295 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0371401 s, 28.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0391807 s, 26.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0413525 s, 25.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0643698 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0517834 s, 20.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0448513 s, 23.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0516995 s, 20.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0722134 s, 14.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.075212 s, 13.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.059152 s, 17.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.078036 s, 13.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0697209 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0711392 s, 14.7 MB/s removed '/mnt/lustre/d77jb.sanityn/nrs_r_oleg120-client.virtnet' removed '/mnt/lustre/d77jb.sanityn/nrs_w_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77jb.sanityn' cpu_npartitions on ost1 is 1 running as uid/gid/euid/egid 500/0/500/0, groups: 0 [touch] [/mnt/lustre/d77jb.sanityn/tbf] Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77jb.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.9483 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77jb.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.8851 s, 5.3 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77jb.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ost_gid ost.OSS.ost_io.nrs_policies=fifo PASS 77jb (135s) == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 22:16:51 (1746152211) ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_tbf_rule=start ext_w jobid={dd.500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r jobid={dd.500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.456534 s, 36.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0298967 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0601145 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0804273 s, 13.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0899271 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0923216 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0983652 s, 10.7 MB/s 1048576 bytes (1.0 MB) copied, 0.113455 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.119269 s, 8.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.132358 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.140617 s, 7.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.153043 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.155449 s, 6.7 MB/s 1048576 bytes (1.0 MB) copied, 0.135661 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.144877 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.223682 s, 4.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.22189 s, 4.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0578706 s, 18.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00372598 s, 281 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00319249 s, 328 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0511272 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0947066 s, 11.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00221236 s, 474 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0704835 s, 14.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.080068 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0673291 s, 15.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0717839 s, 14.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0959334 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0339923 s, 30.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0669898 s, 15.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317271 s, 33.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0301558 s, 34.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0203324 s, 51.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg120-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 6.20857 s, 16.9 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.83661 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext_w nid={0@lo 192.168.201.20@tcp}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_r nid={0@lo 192.168.201.20@tcp}&opcode={ost_read} rate=10 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.36412 s, 46.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328855 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307186 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0226543 s, 46.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.104637 s, 10.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.117686 s, 8.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0896883 s, 11.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.105646 s, 9.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142903 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.112384 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.133466 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.120688 s, 8.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157045 s, 6.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.137467 s, 7.6 MB/s 1048576 bytes (1.0 MB) copied, 0.17026 s, 6.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.178377 s, 5.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.23068 s, 4.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0352267 s, 29.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0310528 s, 33.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0315197 s, 33.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.032438 s, 32.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0708259 s, 14.8 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0364763 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0358177 s, 29.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0423252 s, 24.8 MB/s 1048576 bytes (1.0 MB) copied, 0.0420002 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0416893 s, 25.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.03754 s, 27.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0400902 s, 26.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0418913 s, 25.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0380571 s, 27.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0398666 s, 26.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0317345 s, 33.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg120-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 7.06271 s, 14.8 MB/s Write runtime is 8 s, speed is 12.500000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.83647 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_w ost.OSS.ost_io.nrs_tbf_rule=stop ext_r ost.OSS.ost_io.nrs_tbf_rule=start ext nid={0@lo 192.168.201.20@tcp}&jobid={dd.500} rate=20 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.389861 s, 43.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0382071 s, 27.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0449545 s, 23.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.131405 s, 8.0 MB/s 1048576 bytes (1.0 MB) copied, 0.145418 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14555 s, 7.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.130591 s, 8.0 MB/s 1048576 bytes (1.0 MB) copied, 0.112001 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.129531 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.145425 s, 7.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.123809 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.137371 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.124045 s, 8.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.133099 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.176169 s, 6.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.182529 s, 5.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.174307 s, 6.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318156 s, 33.0 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0230385 s, 45.5 MB/s , 0.0692061 s, 15.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0643849 s, 16.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00222754 s, 471 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0601767 s, 17.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0606335 s, 17.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0562749 s, 18.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0698402 s, 15.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0534774 s, 19.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0638901 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0812947 s, 12.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0556045 s, 18.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.019772 s, 53.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0181934 s, 57.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0224807 s, 46.6 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg120-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 6.92276 s, 15.1 MB/s Write runtime is 8 s, speed is 12.500000 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.99285 s, 21.0 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext ost.OSS.ost_io.nrs_tbf_rule=start ext_a jobid={dd.500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b jobid={dd.500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.54124 s, 31.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0175499 s, 59.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0495587 s, 21.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0342026 s, 30.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0676233 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.107425 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0551915 s, 19.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.136816 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.138222 s, 7.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.140984 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.106727 s, 9.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.196394 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.144453 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.207191 s, 5.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.21164 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.157912 s, 6.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.225371 s, 4.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0127392 s, 82.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0114821 s, 91.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00185496 s, 565 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00393831 s, 266 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0256748 s, 40.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00236734 s, 443 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0769255 s, 13.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0828973 s, 12.6 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0644261 s, 16.3 MB/s 1048576 bytes (1.0 MB) copied, 0.0331603 s, 31.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00726014 s, 144 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0355198 s, 29.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0879354 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0350769 s, 29.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0318816 s, 32.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0263192 s, 39.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg120-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 10.1714 s, 10.3 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.82672 s, 10.7 MB/s Read runtime is 10 s, speed is 10.000000 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 6.38638 s, 16.4 MB/s Write runtime is 7 s, speed is 14.285714 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.81973 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_tbf_rule=start ext_ug uid={500}&gid={500} rate=5 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.922322 s, 18.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0461073 s, 22.7 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0899665 s, 11.7 MB/s , 0.0949023 s, 11.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.129202 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0961747 s, 10.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.128393 s, 8.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.135498 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.116613 s, 9.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.120696 s, 8.7 MB/s , 0.113612 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152698 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.170835 s, 6.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.27528 s, 3.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.330007 s, 3.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.302374 s, 3.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.314759 s, 3.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00202986 s, 517 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0479286 s, 21.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00675962 s, 155 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0498335 s, 21.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0919852 s, 11.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0129755 s, 80.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0981291 s, 10.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.101451 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.102173 s, 10.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.082617 s, 12.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.060573 s, 17.3 MB/s , 0.0521938 s, 20.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0843209 s, 12.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.110097 s, 9.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0658884 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0561494 s, 18.7 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 5, read rate: 5 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.7545 s, 5.3 MB/s Write runtime is 21 s, speed is 4.761904 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 19.556 s, 5.4 MB/s Read runtime is 21 s, speed is 4.761904 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_ug ost.OSS.ost_io.nrs_tbf_rule=start ext_uw uid={500}&opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_ur uid={500}&opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.497462 s, 33.7 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.017714 s, 59.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0397544 s, 26.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0387651 s, 27.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0367538 s, 28.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0510518 s, 20.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.09956 s, 10.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.152929 s, 6.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.139354 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147288 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.162341 s, 6.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.142042 s, 7.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.191787 s, 5.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.196659 s, 5.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.201234 s, 5.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.216639 s, 4.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.207969 s, 5.0 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00222589 s, 471 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00207304 s, 506 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00288371 s, 364 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00545788 s, 192 MB/s , 0.00456597 s, 230 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00250178 s, 419 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0589887 s, 17.8 MB/s , 0.0636889 s, 16.5 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0642156 s, 16.3 MB/s , 0.00651782 s, 161 MB/s 1048576 bytes (1.0 MB) copied, 0.0403537 s, 26.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0502389 s, 20.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0569963 s, 18.4 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0338811 s, 30.9 MB/s , 0.0483196 s, 21.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.024955 s, 42.0 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg120-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 7.6973 s, 13.6 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.84934 s, 10.6 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_uw ost.OSS.ost_io.nrs_tbf_rule=stop ext_ur ost.OSS.ost_io.nrs_tbf_rule=start ext_a uid={500},opcode={ost_write} rate=20 ost.OSS.ost_io.nrs_tbf_rule=start ext_b uid={500},opcode={ost_read} rate=10 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet] [bs=1M] [count=16] 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.476898 s, 35.2 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_w] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0321325 s, 32.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0344039 s, 30.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0701971 s, 14.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0773821 s, 13.6 MB/s 1048576 bytes (1.0 MB) copied, 0.0901601 s, 11.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0758641 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0800974 s, 13.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0754594 s, 13.9 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0819216 s, 12.8 MB/s , 0.0849278 s, 12.3 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0880016 s, 11.9 MB/s , 0.0937998 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.133306 s, 7.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.130246 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.143559 s, 7.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.14344 s, 7.3 MB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [declare] [-a] [pids_r] execvp fails running declare (13): Permission denied 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0292862 s, 35.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00339405 s, 309 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00377768 s, 278 MB/s 1048576 bytes (1.0 MB) copied, 0.0323879 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00168943 s, 621 MB/s , 0.035975 s, 29.1 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.00571372 s, 184 MB/s , 0.027968 s, 37.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0281384 s, 37.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0316082 s, 33.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0323952 s, 32.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00611268 s, 172 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1048576 bytes (1.0 MB) copied, 0.0250385 s, 41.9 MB/s , 0.0346963 s, 30.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0328413 s, 31.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.011679 s, 89.8 MB/s removed '/mnt/lustre/d77k.sanityn/nrs_r_oleg120-client.virtnet' removed '/mnt/lustre/d77k.sanityn/nrs_w_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77k.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 10, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 10.0718 s, 10.4 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.81388 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 10 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77k.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.91729 s, 10.6 MB/s Write runtime is 11 s, speed is 9.090909 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77k.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 9.82192 s, 10.7 MB/s Read runtime is 11 s, speed is 9.090909 IOPS cleanup_tbf /mnt/lustre/d77k.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop ext_a ost.OSS.ost_io.nrs_tbf_rule=stop ext_b ost.OSS.ost_io.nrs_policies=fifo PASS 77k (387s) == sanityn test 77l: check the output of NRS policies for generic TBF ========================================================== 22:23:18 (1746152598) ost.OSS.ost_io.nrs_policies=tbf nid ost.OSS.ost_io.nrs_policies=tbf ost.OSS.ost_io.nrs_policies=fifo PASS 77l (7s) == sanityn test 77m: check NRS Delay slows write RPC processing ========================================================== 22:23:25 (1746152605) ost.OSS.ost_io.nrs_policies=delay ost.OSS.ost_io.nrs_delay_min=4 ost.OSS.ost_io.nrs_delay_max=4 ost.OSS.ost_io.nrs_delay_pct=100 running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg120-client.virtnet] [bs=1M] [count=1] [oflag=direct] [conv=fdatasync] 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 3.25076 s, 323 kB/s running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77m.sanityn/nrs_delay_oleg120-client.virtnet] [bs=1M] [count=10] [oflag=direct] [conv=fdatasync] 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 40.0143 s, 262 kB/s ost.OSS.ost_io.nrs_policies=fifo PASS 77m (53s) == sanityn test 77n: check wildcard support for TBF JobID NRS policy ========================================================== 22:24:18 (1746152658) ost.OSS.ost_io.nrs_policies=tbf jobid ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={*.500} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.206336 s, 81.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0132921 s, 78.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0307256 s, 34.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0362553 s, 28.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0931267 s, 11.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0992748 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0967268 s, 10.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0991051 s, 10.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.10364 s, 10.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.130812 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.124865 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.126737 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.125341 s, 8.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.136537 s, 7.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.139424 s, 7.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.134945 s, 7.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.165455 s, 6.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0573073 s, 18.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00184123 s, 569 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.033288 s, 31.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0680962 s, 15.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.101094 s, 10.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0760516 s, 13.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0464694 s, 22.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0365956 s, 28.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0333487 s, 31.4 MB/s , 0.0298486 s, 35.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0253576 s, 41.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0191657 s, 54.7 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00734325 s, 143 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0140792 s, 74.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0434482 s, 24.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0724272 s, 14.5 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg120-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/dev/zero] [of=/mnt/lustre/d77n.sanityn/tbf] [bs=1M] [count=100] [oflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 8.38067 s, 12.5 MB/s Write runtime is 9 s, speed is 11.111111 IOPS Verify the read rate is under TBF control running as uid/gid/euid/egid 500/500/500/500, groups: 500 [dd] [if=/mnt/lustre/d77n.sanityn/tbf] [of=/dev/null] [bs=1M] [count=100] [iflag=direct] 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.98082 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_tbf_rule=start dd_runas jobid={dd.*} rate=20 16+0 records in 16+0 records out 16777216 bytes (17 MB) copied, 0.397928 s, 42.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0275933 s, 38.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0338677 s, 31.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0882389 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0955257 s, 11.0 MB/s , 0.0937203 s, 11.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.111369 s, 9.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.114024 s, 9.2 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.113796 s, 9.2 MB/s , 0.112956 s, 9.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.12683 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.126586 s, 8.3 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.128189 s, 8.2 MB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.128041 s, 8.2 MB/s 1048576 bytes (1.0 MB) copied, 0.129999 s, 8.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.130491 s, 8.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.147211 s, 7.1 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00222498 s, 471 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0409854 s, 25.6 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00206749 s, 507 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00811607 s, 129 MB/s , 0.0381548 s, 27.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00212835 s, 493 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00448976 s, 234 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.00236082 s, 444 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0729167 s, 14.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0638545 s, 16.4 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0662945 s, 15.8 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0657764 s, 15.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0881753 s, 11.9 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0677732 s, 15.5 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0101624 s, 103 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.0675937 s, 15.5 MB/s removed '/mnt/lustre/d77n.sanityn/nrs_r_oleg120-client.virtnet' removed '/mnt/lustre/d77n.sanityn/nrs_w_oleg120-client.virtnet' removed directory: '/mnt/lustre/d77n.sanityn' cpu_npartitions on ost1 is 1 Limited write rate: 20, read rate: 20 Verify the write rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 7.15686 s, 14.7 MB/s Write runtime is 8 s, speed is 12.500000 IOPS Verify the read rate is under TBF control 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 4.97295 s, 21.1 MB/s Read runtime is 6 s, speed is 16.666666 IOPS cleanup_tbf /mnt/lustre/d77n.sanityn sleep 5 for ZFS MDS sleep 5 for ZFS OST Waiting for MDT destroys to complete ost.OSS.ost_io.nrs_tbf_rule=stop dd_runas ost.OSS.ost_io.nrs_policies=fifo PASS 77n (79s) == sanityn test 77o: Changing rank should not panic ====== 22:25:37 (1746152737) mds.MDS.mdt.nrs_policies=tbf nid mds.MDS.mdt.nrs_tbf_rule=start name nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=start name1 nid={192.168.*.*@tcp} rate=10000 mds.MDS.mdt.nrs_tbf_rule=change name1 rank=name mds.MDS.mdt.nrs_tbf_rule=stop name mds.MDS.mdt.nrs_policies=fifo PASS 77o (8s) == sanityn test 77q: Parallel TBF rule definitions should not panic ========================================================== 22:25:45 (1746152745) mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_tbf_rule=start rule77q_1 uid={ 500 11 3}&gid={500 10 33 100 } rate=100 mds.MDS.mdt.nrs_tbf_rule=start rule77q_2 uid={1000}&gid={1000} rate=100 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_2 mds.MDS.mdt.nrs_tbf_rule=stop rule77q_1 mds.MDS.mdt.nrs_policies=fifo PASS 77q (67s) == sanityn test 77p: Check validity of rule names for TBF policies ========================================================== 22:26:52 (1746152812) mds.MDS.mdt.nrs_policies=tbf oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_overflo uid={500} rate=500: File name too long oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_overflo uid={500} rate=500': File name too long pdsh@oleg120-client: oleg120-server: ssh exited with exit code 36 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test_77p_empty: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test_77p_empty': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p@spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p@spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p.spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p.spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p~spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p~spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p#spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p#spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p/spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p/spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p^spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p^spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p%spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p%spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p*spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p*spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p;spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p;spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p,spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p,spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p?spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p?spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77pspec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p>spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p:spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p:spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p+spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p+spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p=spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p=spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p)spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p)spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p(spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p(spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p{spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p{spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p}spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p}spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p|spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p|spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p[spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p[spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p]spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p]spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p!spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p!spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p&spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p&spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p$spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p$spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p`spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p`spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/mds/MDS/mdt/nrs_tbf_rule=start test77p\spec uid={500} rate=500: Invalid argument oleg120-server: error: set_param: setting 'mds/MDS/mdt/nrs_tbf_rule'='start test77p\spec uid={500} rate=500': Invalid argument pdsh@oleg120-client: oleg120-server: ssh exited with exit code 22 mds.MDS.mdt.nrs_policies=fifo PASS 77p (23s) == sanityn test 77r: Change type of tbf policy at run time ========================================================== 22:27:15 (1746152835) mds.MDS.mdt.nrs_policies=tbf uid mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf gid mds.MDS.mdt.nrs_tbf_rule=start md_rule gid={500} rate=1 mds.MDS.mdt.nrs_policies=tbf jobid mds.MDS.mdt.nrs_tbf_rule=start md_rule jobid={*.500} rate=1 mds.MDS.mdt.nrs_policies=tbf mds.MDS.mdt.nrs_tbf_rule=start md_rule uid={500} rate=1 mds.MDS.mdt.nrs_policies=fifo check the number of requests in queue: fifo: 0 crrn: 0 tbf: 0 delay: 0 fifo: 0 crrn: 0 tbf: 0 delay: 0 cleanup 77r 21318 kill md_thread xargs PASS 77r (44s) == sanityn test 78: Enable policy and specify tunings right away ========================================================== 22:27:59 (1746152879) oleg120-server: error: set_param: setting /sys/kernel/debug/lustre/ost/OSS/ost_io/nrs_orr_quantum=1: No such device oleg120-server: error: set_param: setting 'ost/OSS/*/nrs_orr_quantum'='1': No such device pdsh@oleg120-client: oleg120-server: ssh exited with exit code 19 ost.OSS.ost_io.nrs_policies=orr ost.OSS.ost_io.nrs_policies=fifo PASS 78 (6s) == sanityn test 79: xattr: intent error ================== 22:28:05 (1746152885) File: '/mnt/lustre/d79.sanityn' Size: 512 Blocks: 1 IO Block: 1048576 directory Device: 2c54f966h/743766374d Inode: 144115205272505743 Links: 2 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-01 22:28:07.000000000 -0400 Modify: 2025-05-01 22:28:07.000000000 -0400 Change: 2025-05-01 22:28:07.000000000 -0400 Birth: - fail_val=0 fail_loc=0x80000160 fail_val=0 fail_loc=0x80000131 # file: mnt/lustre/d79.sanityn trusted.name1="value1" PASS 79 (16s) == sanityn test 80a: migrate directory when some children is being opened ========================================================== 22:28:21 (1746152901) SKIP: sanityn test_80a needs >= 2 MDTs SKIP 80a (2s) == sanityn test 80b: Accessing directory during migration ========================================================== 22:28:23 (1746152903) SKIP: sanityn test_80b needs >= 2 MDTs SKIP 80b (2s) == sanityn test 81a: rename and stat under striped directory ========================================================== 22:28:25 (1746152905) SKIP: sanityn test_81a needs >= 2 MDTs SKIP 81a (2s) == sanityn test 81b: rename under striped directory doesn't deadlock ========================================================== 22:28:27 (1746152907) SKIP: sanityn test_81b We need at least 2 MDTs for this test SKIP 81b (2s) == sanityn test 81c: rename revoke LOOKUP lock for remote object ========================================================== 22:28:29 (1746152909) SKIP: sanityn test_81c needs >= 4 MDTs SKIP 81c (2s) == sanityn test 81d: parallel rename file cross-dir on same MDT ========================================================== 22:28:31 (1746152911) - open/close 985 (time 1746152924.87 total 10.00 last 98.47) - open/close 2024 (time 1746152934.88 total 20.02 last 103.76) total: 2500 open/close in 24.86 seconds: 100.58 ops/second snapshot_time 1746152996.959345519 secs.nsecs start_time 1746152914.515879113 secs.nsecs elapsed_time 82.443466406 secs.nsecs open 2505 samples [usecs] 103 14576 2232186 3726448518 close 2504 samples [usecs] 97 5263 818409 633687907 mknod 2501 samples [usecs] 274 14498 1845675 2634230815 unlink 2500 samples [usecs] 403 35736 3486211 13105831987 mkdir 2 samples [usecs] 819 911 1730 1500682 rmdir 2 samples [usecs] 6256 10431 16687 147943297 rename 2500 samples [usecs] 263 75535 10500414 93787892216 getattr 3763 samples [usecs] 4 6797 154141 116527855 parallel_rename_file 2500 samples [usecs] 263 75535 10500414 93787892216 crossdir_rename 2500 samples [usecs] 263 75535 10500414 93787892216 PASS 81d (88s) == sanityn test 82: fsetxattr and fgetxattr on orphan files ========================================================== 22:29:59 (1746152999) multiop /mnt/lustre/f82.sanityn vO_ac TMPPIPE=/tmp/multiop_open_wait_pipe.7050 multiop /mnt/lustre2/f82.sanityn vo_Ac TMPPIPE=/tmp/multiop_open_wait_pipe.7050 PASS 82 (4s) == sanityn test 83: access striped directory while it is being created/unlinked ========================================================== 22:30:03 (1746153003) SKIP: sanityn test_83 needs >= 2 MDTs SKIP 83 (3s) == sanityn test 84: 0-nlink race in lu_object_find() ===== 22:30:06 (1746153006) fail_loc=0x8000060b PASS 84 (10s) == sanityn test 85: Lustre API root cache race =========== 22:30:16 (1746153016) Starting: llapi_root_test /mnt/lustre/f85.sanityn /mnt/lustre2/f85.sanityn Starting test test1 at 1746153017 Finishing test test1 at 1746153020 PASS 85 (7s) == sanityn test 90: open/create and unlink striped directory ========================================================== 22:30:23 (1746153023) SKIP: sanityn test_90 needs >= 2 MDTs SKIP 90 (3s) == sanityn test 91: chmod and unlink striped directory === 22:30:26 (1746153026) SKIP: sanityn test_91 needs >= 2 MDTs SKIP 91 (2s) == sanityn test 92: create remote directory under orphan directory ========================================================== 22:30:28 (1746153028) SKIP: sanityn test_92 needs >= 2 MDTs SKIP 92 (3s) == sanityn test 93: alloc_rr should not allocate on same ost ========================================================== 22:30:31 (1746153031) fail_loc=0x00000163 fail_loc=0x0 /mnt/lustre/f93.sanityn-1/file1 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 5103 0x13ef 0x280000400 0 5121 0x1401 0x240000400 /mnt/lustre2/f93.sanityn-2/file2 lmm_stripe_count: 2 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5122 0x1402 0x240000400 1 5104 0x13f0 0x280000400 rc1=0 and rc2=0 PASS 93 (12s) == sanityn test 94: signal vs CP callback race =========== 22:30:43 (1746153043) osc.lustre-OST0000-osc-ffff8800aa215800.idle_timeout=0 osc.lustre-OST0000-osc-ffff8800b6e70800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800aa215800.idle_timeout=0 osc.lustre-OST0001-osc-ffff8800b6e70800.idle_timeout=0 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.215747 s, 38.0 kB/s fail_val=5 fail_loc=0x80000312 fail_val=6 fail_loc=0x80000329 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 3.02515 s, 1.4 kB/s /home/green/git/lustre-release/lustre/tests/sanityn.sh: line 5764: 3976 Alarm clock dd if=/dev/zero of=$DIR/$tfile conv=notrunc oflag=append bs=4k count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0648 s, 63.2 kB/s osc.lustre-OST0000-osc-ffff8800aa215800.idle_timeout=debug osc.lustre-OST0000-osc-ffff8800b6e70800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800aa215800.idle_timeout=debug osc.lustre-OST0001-osc-ffff8800b6e70800.idle_timeout=debug PASS 94 (13s) == sanityn test 95a: Check readpage() on a page that was removed from page cache ========================================================== 22:30:57 (1746153057) llite.lustre-ffff8800aa215800.fast_read=0 llite.lustre-ffff8800b6e70800.fast_read=0 2+0 records in 2+0 records out 2097152 bytes (2.1 MB) copied, 0.054096 s, 38.8 MB/s fail_loc=0x80001421 fail_val=7 fail_loc=0x80001422 fail_val=10 llite.lustre-ffff8800aa215800.fast_read=1 llite.lustre-ffff8800b6e70800.fast_read=1 PASS 95a (22s) == sanityn test 95b: Check readpage() on a page that is no longer uptodate ========================================================== 22:31:19 (1746153079) llite.lustre-ffff8800aa215800.fast_read=0 llite.lustre-ffff8800b6e70800.fast_read=0 1+0 records in 1+0 records out 12288 bytes (12 kB) copied, 0.00754523 s, 1.6 MB/s fail_loc=0x80001424 fail_val=5 fail_loc=0 llite.lustre-ffff8800aa215800.fast_read=1 llite.lustre-ffff8800b6e70800.fast_read=1 PASS 95b (11s) == sanityn test 100a: DoM: glimpse RPCs for stat without IO lock (DoM only file) ========================================================== 22:31:30 (1746153090) SKIP: sanityn test_100a Reserved for glimpse-ahead SKIP 100a (3s) == sanityn test 100b: DoM: no glimpse RPC for stat with IO lock (DoM only file) ========================================================== 22:31:33 (1746153093) 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00445188 s, 920 kB/s /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK /mnt/lustre/d100b.sanityn/dom has type file OK /mnt/lustre/d100b.sanityn/dom has size 4096 OK PASS 100b (4s) == sanityn test 100c: DoM: write vs stat without IO lock (combined file) ========================================================== 22:31:38 (1746153098) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.101404 s, 20.7 MB/s /mnt/lustre/d100c.sanityn/dom has type file OK /mnt/lustre/d100c.sanityn/dom has size 2097152 OK PASS 100c (5s) == sanityn test 100d: DoM: write+truncate vs stat without IO lock (combined file) ========================================================== 22:31:43 (1746153103) 1+0 records in 1+0 records out 2097152 bytes (2.1 MB) copied, 0.0655477 s, 32.0 MB/s /mnt/lustre/d100d.sanityn/dom has type file OK /mnt/lustre/d100d.sanityn/dom has size 4096 OK PASS 100d (4s) == sanityn test 100e: DoM: read on open and file size ==== 22:31:47 (1746153107) 1+0 records in 1+0 records out 12000 bytes (12 kB) copied, 0.00328885 s, 3.6 MB/s PASS 100e (4s) == sanityn test 101a: Discard DoM data on unlink ========= 22:31:51 (1746153111) /mnt/lustre/f101a.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00689392 s, 594 kB/s PASS 101a (5s) == sanityn test 101b: Discard DoM data on rename ========= 22:31:56 (1746153116) /mnt/lustre/f101b.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.00281317 s, 1.5 MB/s PASS 101b (5s) == sanityn test 101c: Discard DoM data on close-unlink === 22:32:01 (1746153121) /mnt/lustre/f101c.sanityn has type file OK 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.0013488 s, 3.0 MB/s PASS 101c (5s) SKIP: sanityn test_102 skipping ALWAYS excluded test 102 == sanityn test 103: Test size correctness with lockahead ========================================================== 22:32:08 (1746153128) fail_loc=0x415 fail_loc=0x415 fail_val=2 Incorrect size expected (no glimpse fix): Starting test test23 at 1746153130 lockahead_test: lockahead_test.c:1191: test23: assertion 'sb.st_size == sb2.st_size' failed: size on /mnt/lustre/d103.sanityn/f103.sanityn and /mnt/lustre2/d103.sanityn/f103.sanityn differs: 0 vs 1048576 fail_loc=0 fail_loc=0x214 fail_val=2 Starting test test23 at 1746153133 Finishing test test23 at 1746153133 PASS 103 (11s) == sanityn test 104: Verify that MDS stores atime/mtime/ctime during close ========================================================== 22:32:19 (1746153139) SKIP: sanityn test_104 ldiskfs only test SKIP 104 (2s) == sanityn test 105: Glimpse and lock cancel race ======== 22:32:21 (1746153141) fail_loc=0x416 PASS 105 (25s) == sanityn test 106a: Verify the btime via statx() ======= 22:32:46 (1746153166) SKIP: sanityn test_106a Test only for ldiskfs and statx() supported SKIP 106a (2s) == sanityn test 106b: Glimpse RPCs test for statx ======== 22:32:48 (1746153168) SKIP: sanityn test_106b statx() only test SKIP 106b (2s) == sanityn test 106c: Verify statx attributes mask ======= 22:32:50 (1746153170) SKIP: sanityn test_106c statx() only test SKIP 106c (3s) == sanityn test 107a: Basic grouplock conflict =========== 22:32:53 (1746153173) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.15677 s, 66.9 MB/s /mnt/lustre/f107a.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 1 obdidx objid objid group 1 5109 0x13f5 0x280000400 multiop /mnt/lustre/f107a.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 multiop /mnt/lustre2/f107a.sanityn vO_G16022000r10g16022000c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 First grouplock blocks second one PASS 107a (7s) == sanityn test 107b: Grouplock is added to the head of waiting list ========================================================== 22:33:00 (1746153180) 10+0 records in 10+0 records out 10485760 bytes (10 MB) copied, 0.147229 s, 71.2 MB/s /mnt/lustre/f107b.sanityn lmm_stripe_count: 1 lmm_stripe_size: 4194304 lmm_pattern: raid0 lmm_layout_gen: 0 lmm_stripe_offset: 0 obdidx objid objid group 0 5130 0x140a 0x240000400 multiop /mnt/lustre/f107b.sanityn vOG14091995_g14091995c TMPPIPE=/tmp/multiop_open_wait_pipe.7050 Grouplock blocks IO First grouplock blocks second one Second grouplock blocks IO PASS 107b (11s) == sanityn test 108a: lseek: parallel updates ============ 22:33:11 (1746153191) 1+0 records in 1+0 records out 10000 bytes (10 kB) copied, 0.0039507 s, 2.5 MB/s fail_val=4 fail_loc=0x80000414 1+0 records in 1+0 records out 8388608 bytes (8.4 MB) copied, 4.68486 s, 1.8 MB/s PASS 108a (9s) == sanityn test 109: Race with several mount instances on 1 node ========================================================== 22:33:20 (1746153200) 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Starting race between client mount instances (50 iterations): Iteration 1 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 3 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 4 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 5 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 6 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 7 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 8 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 9 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 10 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 11 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 12 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 13 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 14 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 15 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 16 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 17 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 18 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 19 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_refcnt: could not read integer from '/sys/module/intel_rapl/refcnt': 'No such device' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 20 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 21 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 22 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 23 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 24 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 25 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 26 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 27 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 28 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 29 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 30 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 31 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 32 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 33 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 34 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 35 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 36 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 37 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 38 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 39 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 40 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 41 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 42 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 43 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 44 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 45 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory Iteration 46 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 47 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Iteration 48 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/intel_rapl/holders': No such file or directory Iteration 49 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' libkmod: kmod_module_get_holders: could not open '/sys/module/pcc_cpufreq/holders': No such file or directory ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions oleg120-server: libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory Iteration 50 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre 192.168.201.120@tcp:/lustre /mnt/lustre lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 192.168.201.120@tcp:/lustre /mnt/lustre2 lustre rw,checksum,noencrypt,flock,lazystatfs,lruresize,nolock,statfs_project,nouser_fid2path,user_xattr,verbose 0 0 Stopping client oleg120-client.virtnet /mnt/lustre (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 (opts:) Loading modules from /home/green/git/lustre-release/lustre detected 4 online CPUs by sysfs MODOPTS_LIBCFS= Force libcfs to create 2 CPU partitions libkmod: kmod_module_get_holders: could not open '/sys/module/acpi_cpufreq/holders': No such file or directory ../libcfs/libcfs/libcfs options: 'cpu_npartitions=2' ptlrpc/ptlrpc options: 'lbug_on_grant_miscount=1' quota/lquota options: 'hash_lqs_cur_bits=3' loading modules on: 'oleg120-server' oleg120-server: oleg120-server.virtnet: executing load_modules_local oleg120-server: Loading modules from /home/green/git/lustre-release/lustre oleg120-server: detected 4 online CPUs by sysfs oleg120-server: MODOPTS_LIBCFS= oleg120-server: Force libcfs to create 2 CPU partitions Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre Starting client: oleg120-client.virtnet: -o user_xattr,flock 192.168.201.120@tcp:/lustre /mnt/lustre2 PASS 109 (735s) == sanityn test 110: do not grant another lock on resend ========================================================== 22:45:35 (1746153935) fail_loc=0x525 fail_val=3 fail_loc=0x534 File: '/mnt/lustre/f110.sanityn' Size: 0 Blocks: 1 IO Block: 4194304 regular empty file Device: 2c54f966h/743766374d Inode: 144115205306056707 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2025-05-01 22:45:36.000000000 -0400 Modify: 2025-05-01 22:45:36.000000000 -0400 Change: 2025-05-01 22:45:36.000000000 -0400 Birth: - fail_loc=0 fail_val=0 Connected clients: oleg120-client.virtnet oleg120-client.virtnet PASS 110 (66s) == sanityn test 111: A racy rename/link an open file should not cause fs corruption ========================================================== 22:46:41 (1746154001) SKIP: sanityn test_111 needs >= 2 MDTs SKIP 111 (2s) == sanityn test 112: update max-inherit in default LMV === 22:46:43 (1746154003) SKIP: sanityn test_112 We need at least 2 MDTs for this test SKIP 112 (2s) == sanityn test 113: check servers of specified fs ======= 22:46:45 (1746154005) lustre-OST0000-osc-ffff8800b3a9a000 active. lustre-OST0001-osc-ffff8800b3a9a000 active. lustre-OST0000-osc-ffff8800b3a9a000 active. lustre-OST0001-osc-ffff8800b3a9a000 active. MGC192.168.201.120@tcp active. PASS 113 (4s) == sanityn test 114: implicit default LMV inherit ======== 22:46:50 (1746154010) SKIP: sanityn test_114 We need at least 2 MDTs for this test SKIP 114 (3s) == sanityn test 115: ldiskfs doesn't check direntry for uniqueness ========================================================== 22:46:52 (1746154012) SKIP: sanityn test_115 ldiskfs only test SKIP 115 (2s) == sanityn test 116: DNE: Set default LMV layout from a remote client ========================================================== 22:46:54 (1746154014) SKIP: sanityn test_116 needs >= 2 MDTs SKIP 116 (2s) == sanityn test 200: service remains healthy while able to process request ========================================================== 22:46:56 (1746154016) at_max=10 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=20 fail_loc=0x1e1 fail_val=20 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1939 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1475 s, 0.2 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1844 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out , 20.1549 s, 0.2 kB/s 4096 bytes (4.1 kB) copied, 20.1473 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1633 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.2137 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 20.1735 s, 0.2 kB/s 4096 bytes (4.1 kB) copied, 20.1789 s, 0.2 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.15 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.2669 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.2573 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.1481 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.3217 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.3085 s, 0.1 kB/s , 40.2569 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.3061 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 40.268 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.2962 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.4601 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.3926 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.4532 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.4414 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.4359 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.4928 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.4182 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 60.4278 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.3839 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.4565 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.4714 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.6113 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.4862 s, 0.1 kB/s 1+0 records in 1+0 records out 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied4096 bytes (4.1 kB) copied, 80.5498 s, 0.1 kB/s , 80.5097 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.5794 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 80.5249 s, 0.1 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.504 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.545 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.594 s, 0.0 kB/s 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 100.682 s, 0.0 kB/s at_max=600 at_unhealthy_factor=3 ost.OSS.ost_io.threads_max=116 oleg120-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8800aa210800.ost_server_uuid 50 osc.lustre-OST0000-osc-ffff8800aa210800.ost_server_uuid in IDLE state after 0 sec PASS 200 (113s) cleanup: ====================================================== == sanityn test complete, duration 7510 sec ============== 22:48:50 (1746154130) sanityn: FAIL: test_16j dio copy 1 w/bsize 1049600 failed: 1 === sanityn: start cleanup 22:48:51 (1746154131) === Stopping clients: oleg120-client.virtnet /mnt/lustre2 (opts:) Stopping client oleg120-client.virtnet /mnt/lustre2 opts: === sanityn: finish cleanup 22:51:59 (1746154319) ===